DAY 1:
NTNU SmallSat: a Hyper-spectral imaging mission (Mariusz)
Goal is to do oceanography
Algae blooms may suffocate fish
And a pipeline of SmallSats
We need to know if what we do is fruitful to ocean study
Many of the oceanographers asked for raw data
Short observable time - several passes but not all are viable.
Questions from the audience
Will there be enough clear sky to get good images?
There will be holes in the clouds so you can see things through it
Use weather forecast to plan where to scan
HSI will allow for other information that you can see with the eye
Need to emphasize that we have to perform trade studies
How do linux systems survive?
Communication trade-offs
Antenna pointing
- Gain
- Power consumption
- Data rate
Cost of the SmallSat?
To get a complete picture incremental updates are needed
Do we specify the satellite performance criteria from the bottom or from the top?
Know the end-user requirements
Hyperspectral remote sensing (João)
Lots of questions regarding the optical characteristics - and comparison to Sentinel
Will be using JPEG 2000 and Idletechs technology compression, multivariate techniques borrowed from the field of chemometrics
The importance of storing the raw data to be able to recalibrate the data is important to be able to get valid data for science.
We want to compress data to be able to downlink it
2nd order effect from Visual bands may enter into NIR bands.
There exist simplistic algorithms for agricultural or land applications (NDVI)
Don’t have HSI with ground truth validation using our camera of chlorophyll.
Need more data to build a database of endmembers (chlorophyll signatures)
Ajit Subramaniam et al.: The tale of three upwelling systems
hydrocarbon gas bubbles from boat and space
Sunglint is good for contrast when looking at sliks
Best angle is when there's sunglint but it's not useful for detecting chrolophyll
Important to have nice ground resolution and time variability
If you average the image in distance and during several days interesting info is lost
- 10 days later everything is different on the surface
Dual laser fluorometer will possibly be available for in situ observations
Gathered a lot of data from the coast of Vietnam, east-sea, falkor cruise 2016
Observing the dynamics of Oceanographic phenomena
Vertical transport hypothesis
With HSI, holy grail to better characterize phytoplanktons, community structure
Trying to develop models for upwelling from hydrocarbon gas, using acoustics, fluerometers, etc.
Milica Orlandić: onboard HSI processing
Possible to build in new state of the art HSI algorithms in the processing pipeline
Zynq board (7000 series as of now)
First Zynq 7020 and later, Zynq 7030
Possible to optimize for power usage and parallelism.
PLS for further optimizing for our specific platform
FPGA for low power consumption, compact size, low weight, real-time processing.
Caveats: low abstraction, deep understanding of design, long design time
Lots of recent work using FPGAs for HSI processing
PCA algorithm
- It keeps the things that vary a lot
- Maybe to get details, we need things that don't vary much
- The algorithm should be customed for the desired application
- FPGAs comes at an engineering cost, but the advantages (may) outway the cost
Image data should be parallelized
Functionalities can be updated on-the-fly
Can not use regular microcontrollers due to limitations (Julian, Milicia)
Should not go for a barebones system as we need the flexibility (Joao)
Need to make the entire pipeline to be able to accurately estimate our power consumption and other crucial parameters.
Implement Idle Techs algorithms to be able see limitations
Need to determine algorithms to be able to adapt it to HW and integration.
There might be certain bands that are more important for certain applications and a maximum variance might not be the way to go, when looking for small changes.
The Idle Tech SW can be adjusted for specific applications (Joao)
What data levels can we expect?
Two pipelines, one for raw data and one for level ?2-1b? data.
Could be changed.
Need to build something in order to test it.
Julian Veisdal: HSI HW status
Off the shelf components camera with 3D printed body
USB interface is fast enough.
Interchangeable and flexible design of camera housing (3D printed)
Based on automotive / industrial parts, to be tested. Not space-tested (SpaceX)
The design is as seen in the slides, ready to be printed for testing.
Need both shock testing and thermal testing as well as good characterizing of the camera behaviour.
Desktop ready is not the same as space ready.
Emlyn Davies: Particle Measurements with help from AI of the ocean
Good presentation and presentation technique.
Measurements is done from 0.05 um to 2000 um, to get all particles
Knows about backscattering and angles
In-situ measruements, imaging micro-organism.
Using 6-layered DCC for classifying (tensorflow based).
Mapping concentration at different depths
May be able to see copepods from space, based on models in-situ measurements
The pictures shown is at 15 MB image at 70 Hz
Most of the processing and analysis is traditional image processing.
Jan Otto Reberg: Intelligent compression for SmallSat HSI mission
There will be a lot data so we should compress it in a smart way.
Idle tech has a lot of experience with compression of HSI or multivariate data
Need to make assumptions about the data
Assumed dependence between different bands i.e. some data is irrelevant
To be compressed: # of known phenomena, noise, outliers, etc.
The model for signal flow is changeable, and is based on ground and on board.
PCA caveats due to variance based approach can be circumvented by prior knowledge.
Want to retrieve information that is useful for scientific purposes, and not necessarily shannon based compression approaches wrt. Information theory.
Need to decide what kind of satellite communication protocol to use.
JPEG2000 is the preferred image compression algorithm. link
Reasonable good compression
Binning will be great for HW, but might be unbeneficial for information recreation.
Use FPGA to be able to test different approaches
Comment: it takes fro granted that we know what there is in the image
- We assume we have chlorophyll in the image spectra
- If we are wrong we can correct it
Comment: sometimes you want high frequency components, others low frequency or average
Tuning parameters should be established
Concerns being raised for this method used for oceanography
Need to test it for water specific applications
Endmember and mixing of pixels may suffer under compression
Will be able to reproduce images due to using the entire spectra.
The compression does not interpret the data, it only compresses
When should the atmospheric correction take place?
Davide Alimonte: Collection and analysis of ocean color bio-optical data.
Good presentation about remote sensing of oceanography basics
See slides in dropbox folder
Conclusion: don’t measure when there’s higher wind speeds of 4m/s or bright sun.
Increasing sampling frequency doesn't necessarily mean more data because there's bias
Contributions to reflectance comes from all over, and not just directly below the sun.
Integration time of sensor have great effect on sensor performance
Quality assurance and validation of gathered data is paramount for oceanographers
Even large expensive satellites need to be recalibrated radiometrically
Comparison of ANN and standard Regression
Rho factor
Depends on wind speed
Has to be estimated to determine sun and sea radiance
30 % accuracy is great wrt. Ocean colour analysis
Atmospheric correction need to be on point, or else the data will be garbage.
The field of Remote sensing is not mature in the field of multivariate analysis (ajit)
Using HSI might be able to better model hyperspectral images.
The performance of neural networks depends on input data
Comment for us:
We have to validate hyperspectral images
We have to partner with Metrology labs
Comment: the summer pictures taken of the river should be analysed
- Maybe what we are getting is not only what river transmits but also reflected light from TX
Trygve Fossum: Adaptive data collection: What robots can do for ocean science
General about sampling the upper oceans, where to sample
Ocean models will tie together all types of sampling devices.
Combining models and sensors to generate sensing strategy to acquire desirable data
Data fusion is still under work
Remote sensing will be able to improve the data from AUVs and ASVs
The sensor equipped AUV is able to adapt its behavior to sample the desired data
Remote sensing depth capabilities depends on what you are looking at, up to 10s meters
Ajit's comment: we should have 2 exact copies of the camera, one on the satellite and another one on a UAV
Should use HSI at several altitudes to confirm atmospheric correction algorithms, and other types of corrections.
Difficult to do properly, need to be done carefully
Olav Godø: Marine ecosystem function and dynamics a playground for advanced technology
Don’t think the right Powerpoint was uploaded to the dropbox folder.
A moving platform will mix the space and time dimension, which needs a model to properly separate. This is difficult if not impossible.
Need to carefully plan data sampling schemes to not regret it in hindsight
Discovers unexpected natural phenomena.
Mackerel is one of the most valuable resources in the Norwegian oceans
Use LIdars for water penetrating observations
Airplanes were expensive, but lidars are now much more commercially available.
Should be used in drones, could be used i future satellites?
We are moving into a regime that is unknown with HSI, which makes it impossible to use a good model to verify, to begin with.
The challenges of observing the ocean is due to the high rate dynamic effects.
Acoustics are the only observations who preserve spatial and temporal considerations.
A lot of data is being used to make and validate models to observe oceanographic events.
Acoustics might be an underused type of sensor in AUVs, Gliders, buoys, etc.
Remote sensing could/should be used for verification and better model development
Difficult to merge data from different types of sensor and sensor schemes.
Validation of models is important.
Satellite observations is important.
Mariusz Grøtte - Detailed Mission design and analysis
30 % of the time it will not be able to harvest enegry, thus energy budget is important
Mission requirements defines several levels of success. See slides
Need to characterize the camera to better know what we can do and see.
Operational requirements using multiple agents is a future
Trade off between spatial resolution and SNR wrt. Focal length
Two ways to do geometric image processing
Use orbit data
Use ground control points
For geometrical data there will be used orbit data, as of now.
We want to be using adaptive sampling techniques in the satellite as well.
Status of the compression pipeline
Idletech has a lot of software used for spectral data earlier, these algorithms will adjusted space use.
Milicia has developed some of it for the HW we are using. (The MEX-algorithm).
Estimated 6 months of development time to make it ready for SmallSat
This is a short time
It is research and will never be complete (TAJ)
Most remote sensing satellites programmes use post-processing, and does not process on board.
Have not set the minimum requirements for processing at this time.
Plan A. Downlink processed data?
Plan B is to use RAW data and post-processing, if the processing fails?
- Oceanographers should say what's raw data for them
It will be possible to update software on orbit, this is complicated and tricky
Bug fixes is complicated, and is not a simple thing to do in space
It needs to be decided how these fixes will be performed.
The range of the sensor has a range of 300 - 1000 nm with less sensitivity at the edges
It should always be under 3 watts power consumption, but need to know more about the compression pipeline (Julian).
Lossless compression will take a lot of time.
Will we be using more energy on compression than we would on down-linking?
Probably not (TAJ)
Current estimate of processing time consumption will be about 2 minutes
We need to tailor our system for the ocean colour community wrt. Type of data
Tradeoff between spatial resolution and swath angles
Possible to get better spatial resolution than Sentinel.
The satellite is not passively stable, and the actuating will be used for stabilizing
Electrical propulsion is cost effective, but not perfectly accurate.
A gimbal may add error
Should do optimization on the camera optics based on SNR.
Can we correct AT&C at the same time we are acquiring images?
Comments on studying propulsion
Where is the star tracker located?
It should be on the outside
Ajit: should be able to look at the moon
A lot of what you see is due to atmospheric effects.
Peak power of 30 watts at satellite, 5 watts peak power on Payload
3 minute operation on peak power ideally
Need to position the S-band antenna in a way that makes it possible to down-link
Our Cubesat provider will be doing system requirements in parallel.
This an ambitious project.
Torbjørn Eltoft - Chlorophyll-a estimation from multispectral remote sensing using ML
CIRFA is doing remote sensing of the artics.
Optical image is useful to verify and validate remote sensed scenes.
Two buoys were used to record data
iSphere
Self Locating Datum Marker buoy
Oil slick model dynamics development using JPL/NASA UAV-SAR, SAR images
Repeating the basics of oceanographic remote sensing
It is not trivial to validate the results you get from remote sensing instruments.
At 5 nm spectral resolution we might be able to distinguish between different species of algae or phytoplankton.
Most of the pixels you will see from a satellite will be mixed.
If they are different enough we might be able differentiate them
You will have more information using more bands
There a limitations to traditional band based algorithms for extracting chlorophyll content
Machine learning can garner interesting results, perhaps even better results
There are different ways to calculate chlorophyll, and more data will create better results.
Stian Solbø - Arctic and Ocean remote sensing with Satellites, Airplanes and drones
NORUT, using UAVs and satellites to gather data
A single SAR image can give information about wind, sea state, ocean surface state etc.
It will be beneficial to use the same payload in the UAV as well as the satellite
Calibration is important to get useful data for the scientific community
NDVI values are the most important for vegetation measurements, can create 3D maps
Use a commercial airplane to gather data, existing approach. A lot of paperwork
Could put our payload on a high altitude (commercial) plane in an attempt to recreate the condition closer to the conditions we expect to see in space.
Dornier DO-228 could be used to calibrate and validate our SmallSat
NLIVE - web portal to interact with data gathered by NORUT
Should combine HSI, radar and models create high level datasets for Oceanographers
Martin Ludvigsen - Challenges and operations in seabed mapping for marine mining, Arctic andarchaeology exploration
Norway might be blessed with marine minerals (between Iceland and Svalbard)
There are no methods for regional mapping for marine mining as of now.
The undersampling is also present when searching for marine mining sites
The presence of the instruments generate a lot of light pollution
Ship wreck detection (and classification?) using HUGIN generating 3D models.
Nansen legacy: interdisciplinary project to get sustainable environments
Jon Harr / Eirik Blindheim - Norwegian SmallSat program – status and strategies
Norsk Romsenter - link between government, industry and ESA
The ocean is important for norway (6/7 of norwegian territory is sea)
The Norwegian government wants to have Norwegian satellites in space.
Challenges can be counteracted with “new space”-strategies, smallsat approach
Even though they have low life expectancy, some satellites far exceeds early estimates
AISSat and NorSat are new space success examples.
NorSat-1: AIS + 2 additional payloads –> S-band for downlink (2 Mbps, average 1Mbps)
NorSat-2: VHF data system (AIS freq), Yagi antenna (<1m)–> S-band for downlink (2 Mbps, average 1Mbps)
A launch of AISSat 3 now on tuesday! Supposed to make contact 9.00 UTC
Comment: AIS to improve attitude is a good idea
Several ground stations used to down-link / communicate with NorSat and AISSat
AISSat and NorSat provide a more complete registration of ship data
The Norwegian Space Center welcomes the HSI SmallSat program
Use AISSat data to improve attitude determination
The main challenges for our project lies in the following
The payload, will it work, will it be able to provide useful data?
Will the ADCS be able to accurately control attitude in a satisfactory manner
Communication will be dependent on a a lot of factors
The data rate will change between every pass
An average data rate is more useful than to say what is possible.
Andreas Nordmo Skauen - FFI cubesat-lab
FFI research areas contain the fields of electronics, rockets and nuclear energy technology. A very multidisciplinary institute.
AISSat program consists of several simple iterations.
Experience show that the performance shown in a datasheet is for ideal situations.
There is a large need for thermal, vibration and vacuum-testing, FFI have experience.
FFI could be a useful asset due to experiences with NorSat and AISSat
Cubesat kits can be useful to “play” around with
GOMspace and other providers have software development kits
There exists 5 day training courses for this (Mariusz was there)
It's difficult to model everything (in 3D and scenarios), should build it and throw it around
Mainly the cables
FFI wants to help us
Could integrate our payload in their kit, maybe
FFI/Andreas is interested in having competence similar to GOMspace at NTNU
We shouldn't lose too much information on-board
It's probable to get a late reply for the tender
FFI create realistic test by
Simulating communication loss
Sun lamps (strong lamps)
Thermal chambers, etc.
Ajit Subramaniam - extra on remote sensing
Be careful about what kind of terms you use when you claim what you are able to see
It is important to remember that we are always observing backscattering
Limits and magnitudes will affect the inverse relationship
CDOM will make it difficult to see low concentration of chlorophyll, not trivial
High concentrations of chlorophyll are more trivial to distinguish
Hyperspectral data will provide useful insight not gained from multispectral
Davide: you should normalise to depend the least on environmental factors such as:
Sun elevation
Atmosphere
Viewing angle
Ajit: we should establish a clear objective: algae blooms or ocean colour or...?
Atmospheric correction is complicated
A lot of the signal will be lost due to atmospheric effects i.e. aerosols
By slewing there might be problems with BRDF
Uncertainties at 30 % is not uncommon for oceanographers.
Discussions and next steps
The derivatives will be great as a HSI
Need high SNR
Algorithms only using the slope will not work well in type 1 waters
Multivariate analysis might give great unknown insight
Need to be compared with traditional methods
Who are we producing this data for?
Clarify the objective.
We want to be relevant to the ocean colour community
It is ambitious
We have have have the flexibility to work on different use cases but need something to work towards
We can get 2 images using 1 camera+1 mirror
Idle tech algorithm will use soft modeling to meet the models of oceanographers
Valuable spatial resolution of oceanographers is anything between [10 … 10 000] meters
SNR will get better by several overpasses, speaks for non-nadir views
Better spatial resolution comes at a cost of the size of the payload
Propulsion is needed for attitude control, if not placed in the correct orbit
There exists polarization filters than can be turned on and off.
Might be useful for atmospheric correction.
Will come at a significant signal strength cost
Saturation is bad, and we will not be able to get good data
Need numbers regarding SNR requirements
What are NASA and other companies referring to when they say SNR
TOA, water reflectance? What wavelengths? All wavelengths?
Second order effects need to be characterized.
It might be valuable to use several cameras
Can replace the slewing
Will cost power consumption
Slewing is not the difficult part
Use different cameras for different wavelengths
During the slew maneuver it is reasonable to expect high accuracy.
Might use mirrors (nah)
Redundancy is a nice thing to have in space.
More cameras is more complex and we should not try to drown ourselves
Could be useful to increase SNR at the cost of spectral resolution
Should consider changing the lens diameter to gain better signal.
Could be done, should be done
- Gyros are extremely good for accuracy, and we need that for our application
All the presentations should be available at some Dropbox folder as of now