# SIT-FUSE Docs SIT-FUSE utilizes self-supervised machine learning (ML) that allows users to segment instances of objects in single and multi-sensor scenes, with minimal human intervention, even in low- and no-label environments. Can be used with image like and non image-like data. Currently, this technology is being used with remotely sensed earth data to identify objects including: * Wildfires and smoke plumes * Harmful algal blooms and their severity * Palm oil farms * Dust and volcanic ash plumes * Inland water bodies Figure 1 depicts the full flow of SIT-FUSE and figures 2 and 3 show segmentation maps and the information extracted for instance tracking across scenes. SIT-FUSE’s innovative multi-sensor fire and smoke segmentation precisely detects anomalous observations from instruments with varying spatial and spectral resolutions. This capability creates a sensor web by incorporating observations from multiple satellite-based and suborbital missions. The ML framework’s output also facilitates smoke plume and fire front tracking, a task currently under development by the SIT-FUSE team.

***

Figure 2. The first row contains scenes from different instruments/instrument sets used as input. The second row shows SIT-FUSE’s output segmentation maps for the input scene, and the third row shows retrieved objects of interest, in this case, fire and smoke.

***

Figure 3. Each 4-image set is generated from a separate GOES-17 scene over an observed fire in 2019. The top row of each set depicts radiances and their associated clustering output from software system S. The second row shows the radiances with an overlay of the subset of clusters assigned to the contexts of smoke and fire. The bottom row shows the input radiances with shape approximations for smoke and fire generated via the openCV contour functionality. The green arrows depict the products that can be used for instance tracking. For cross-instrument instance tracking we will use contrastive learning to map the instance signatures across the different domains.

*** Recent Talks: {% embed url="" %} {% embed url="" %} 2022 ECMWF–ESA Workshop on Machine Learning for Earth Observation and Prediction {% endembed %} {% embed url="" %} 2022 TIES Annual Meeting {% endembed %} References: * LaHaye, N.; Easley, A.; Yun, K.; Lee, H.; Linstead, E.; Garay, M.J.; Kalashnikova, O.V. Development and Application of Self-Supervised Machine Learning for Smoke Plume and Active Fire Identification from the Fire Influence on Regional to Global Environments and Air Quality Datasets. *Remote Sens.* 2025, *17*, 1267. * Lahaye, N., Garay, M. J., Bue, B., El-Askary, H., Linstead, E. “A Quantitative Validation of Multi-Modal Image Fusion and Segmentation for Object Detection and Tracking”. Remote Sens. 2021, 13, 2364. * Lahaye, N., Ott, J., Garay, M. J., El-Askary, H., and Linstead, E., “Multi-modal object tracking and image fusion with unsupervised deep learning,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 12, no. 8, pp. 3056-3066, Aug. 2019, doi: --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://nicks-personal-organization-2.gitbook.io/sit-fuse/readme.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.