SkyServe | Developer Portal

SkyServe

Model Submission

My Models

Model Details

Notifications

Profile

Submit a new model

There are two modes available for submitting the model.

During the model submission phase, helpful validation errors or messages would be provided in case any of the input parameters in the Model Submission form does not adhere to the guideline specified below.

Beginner mode

Receive step-by-step guidance in filling out all the fields with detailed information.

Expert mode

This mode is suitable for experienced users who are familiar with the submission process. Furthermore, when resubmitting a model, the expert mode is automatically selected, allowing users to easily update or adjust the previous submissions without re-entering detailed information. This mode also displays all fields at once on a single page for easy access.

Fields to submit

Model Name :

A unique name for your Model, ensuring it does not exceed 40 characters in length

Model Description :

A model description typically provides a brief overview and relevant details about a specific model. The model description aims to provide users with a clear understanding of what the model does and how it can be utilized effectively

Visibility :

Specify whether you want your models to have Private or Public visibility

Private

Selecting this option means that your models will be visible and accessible only to you as the owner/developer who submitted them. They will not be visible on the marketplace. This is the default visibility of all models submitted on the portal. It is recommended to not switch out of Private unless the model has been exhaustively tested

Public

Opting for this option makes your models visible on the marketplace. They will be able to run your models to obtain insights on the User Portal. This feature is not available currently. Developers will be notified when the Marketplace is live. Making your model Public does not expose your repository, models or scripts to anyone – users on the Marketplace or other developers

Model Repository URL :

The GitHub/Bitbucket URL of the repository containing the model as specified in the guidelines. Please note that only these two platforms are supported

Provide the repository URL where the model and its associated files are stored. A sample repository following the model submission guidelines can be found here

Note : Please check model submission guidelines below to make sure that your repository structure and code adheres to the specification provided.

Model Path :

For Deep Learning (DL) models, please specify the exact location within the model repository that points to the trained model (.onnx file). This field can point to the /Models folder in case of Non-DL models since they won't have any pre-trained model

See Example from here

Auth Key :

An authentication key (Personal Access Token), is a unique alphanumeric code or token that serves as a secure credential for SkyServe to access the repository

To generate Auth Key for GitHub Repo please have a look at here

To generate Auth Key for Bitbucket repo please have a look at here

Model Configs :

If the model parameters be configured by a user (through the User Portal) or by the developer during testing, Model Configs can be used to define these. Any command-line arguments (argparse values) that can be passed to the model inference script would come under this. These arguments allow customization of the model's behavior by adjusting specific parameters. Each argument can be described using the following fields

Argument-name

The name of the configurable parameter.

Datatype

The data type of the argument (int or float).

Min

The minimum allowed value for the argument.

Max

The maximum allowed value for the argument.

Default Value

The default value assigned to the argument if no value is provided by the user.

Example : if the script accepts a '--threshold' argument as an integer, where the threshold parameter is a configurable integer value that sets a threshold for the model. Its minimum value, maximum value and the default value can be provided using the respective fields as described above.

The main inference script can be invoked using the following command : 'python main.py --threshold=8', where <threshold> is a configurable parameter having <int> datatype

Model Input :

The “Input Bands” section requires specifying the band order of images accepted by the model. This information is crucial for a custom script that will feed an image to the model during inference. If the model accepts PNG/JPEG image formats, it can only accommodate RGB bands in the same order. Ensure that the number of bands and band order selected is the same as that accepted by the model. The inference script must not have any pre-processing steps that change the band order. In short, an image as configured in this section should be accepted by the model without any further modifications

Input Product Type :

SkyServe can deliver three different types of image products, each with a different level of processing applied. Please specify the desired level of processing applicable to your use case and the type of input required by the model to work. The system currently offers the following products

Radiometrically Calibrated Scene (L1A)

A band-registered image with Red, Green, Blue, and Near Infrared bands with corrections applied for sensor defects and normalized lighting. The product is made available as a TIFF image optionally accompanied by Cloud, Cloud Shadow and NoData masks.

Haze Compensated Scene (L1B)

This is an L1A product with haze compensation applied. This product allows the processing of scenes that would otherwise fail due to the scattering of light obscuring large parts of an area of interest. Helps enhance a model’s ability to discriminate features.

Geo-referenced Scene (L2A)

An L1A or L1B product that is georeferenced and aligned to true north. This is the product selected for a model if no other level is specified.

Data Mask :

Each product may have (a) clouds, (b) cloud shadows, or (c) Faulty pixels that are left incorrected. Data masks can be requested as input along with the image to be used either for pre-processing the image before inferencing or to post-process the result after inferencing. It is important to note that the corresponding datamask will be delivered as separate single TIFF file for each of the data mask requested. Therefore, please ensure that your model can handle the total number of bands specified as well as the data mask requested

The following data masks can be used to exclude specific pixels from processing

Cloud mask

Denotes presence (1) or absence (0) of cloud at a given pixel

Cloud shadow mask

Denotes presence (1) or absence (0) of a cloud’s shadow at a given pixel

No data mask

Denotes a faulty pixel that has not been corrected through interpolation. It is recommended to exclude such pixels (value = 1) from the final output, i.e. use only pixels with value = 0 in this mask.

Note : If your model accepts <red> and <green> band along with <cloud_mask>, then please specify <red> and <green> as "Input Bands" and <cloud_mask> as "Data Mask"

Model Output :

Please choose the appropriate model output type based on model’s inference from the following categories. For detailed information on each of these output type along with sample examples please refer to Appendix

Image type :

Image type

Select image type as Integer or Float and define the values accordingly.

Integer

An image file with different bands, where each pixel value is represented as an integer.

Float

An image file with multiple bands, where each pixel value is represented as a floating-point number. Each band in the image represents a specific attribute, such as NDMI (Normalized Difference Moisture Index) or NDVI (Normalized Difference Vegetation Index), etc.

Values

Specify the range of pixel values in the image output (Min and Max).

Threshold range

Define the range for a threshold parameter, including the minimum value, maximum value, and precision (default precision is set to medium).

Segmentation :

Segmentation Type

Specify the type of segmentation output.

Binary Mask

A binary mask represented as a TIFF file. The mask consists of pixels with values of either 0 or 1, where 0 represents the background and 1 represents the segmented object.

Multi-class

A multi-class segmentation mask represented as a TIFF file. The mask contains multiple values, where each value represents a specific class or category. The number of different values in the mask corresponds to the number of classes.

Object Detection :

Point

The object detection results in the form of a JSON object. Each object in the “prediction” array includes information such as the prediction score, the x and y coordinates of the detected point.

Bounding Box

The object detection results in the form of a JSON object. Each object in the “prediction” array includes information such as the prediction score, the coordinates of the bounding box (xmin, ymix, xmax, ymax) that tightly encloses the detected object.

Object Classification :

Point

The object classification results in the form of a JSON object. Each object in the "prediction" array includes information such as the class ID, label, prediction score, and the x and y coordinates of the classified point.

Bounding Box

The object classification results in the form of a JSON object. Each object in the "prediction" array includes information such as the class ID, label, prediction score, and the coordinates of the ounding box (xmin, ymin, xmax, ymax) that tightly encloses the classified object.

Model Guidelines

If you are building a Tensorflow model, we require the model to adhere to certain TensorFlow and onnxRuntime versions for better efficiency and performance on our device. Following the guidelines will help us convert the onnx model to respective TensorRT model which will be optimized for the device for better performance.

Note : A similar process to get to the specified onnx version need to be followed for other frameworks like Pytorch, etc.

Library Version Requirements :

The following versions are to be maintained while using the libraries

Tensorflow 2x

Onnx 1.13.0

Tf2onnx 1.13.0

Protobuf 3.20.3

Tensorflow model to ONNX :

You may use the sample colab notebook as a conversion guide here

The typical flow for the model conversion is documented below

Install Tf2onnx-1.13.0 and Onnx-1.13.0

1. pip install git+https://github.com/onnx/tensorflow-onnx

2. OR pip install onnx

Code snippet for onnx conversion

1. input_signature = [tf.TensorSpec([B,H,W,C], tf.float32, name='input_name')]

2. onnx_model, _ = tf2onnx.convert.from_keras(model, input_signature, opset=11)

3. onnx.save(onnx_model, 'model.onnx')

Change the <name> field in above code snippet (1.) to the name of the input node in your model.

1. For existing models, you may use netron to analyze the model and get the input node name and other parameters. The sample notebook also provides an example.

Note : [B,W,H,C] corresponds to Batch, Height, Width, Channel – please make sure to set them according to your model. If a model requires preprocessing steps like splitting an input image to small tiles, it has to be explicitly provided as a preprocessing step in main.py. Check the section: Model Repository Structure

Important Notes

1. The size of any model shared should be less than 200 MB.

2. Use opset11 while converting your tensorflow model to onnx. The same holds for other frameworks.

3. Ensure that the code can work on multiple input images present in the '/input' directory. Kindly ensure to follow the output protocol to match the corresponding input with it's output.

a. [Output Protocol: '{inputfilename}-{suffix}.{extension}'].

b. The inference script runtime output should be stored in '/Runtime' directory while the intermediate outputs from tiling should be stored in the '/Intermediate' directory as described in the respository structure below.

4. Ensure that the operators used in your model are compatible with TensorRT7. If they are are not, a helpful warning or error will be shared.

5. Jetson Framework Versions :

a. The Jetpack Version being used is 4.6

b. CUDA version 10.2 v10.2.89

c. onnxRuntime 1.13.1

d. Pycuda 2022.2.2

Model Repository Structure

The submission of model is done by sharing a open repository containing the models, which will be accessed through the link shared with Skyserve while submission.

All the Models should have the following folder structure

/Models Folder

1. Containing the main model which needs to be run

/SampleData Folder

/Input

Contains Input images/files to test the machine learning / deep learning model.

/Output

Corresponding expected output for the images if the model runs on /Input images.

1. The input file and output file name should follow a specific protocol so that the input image output can be traced back to its corresponding output.

2. [Input Protocol: '{inputfilename}.{extension}'], [Output Protocol: '{inputfilename}-{suffix}.{extension}']

3. Please refer to Appendix for more details.

/Utils Folder

1. Any additional logic required to inference the model apart from the code written in main.py should be kept inside the Utils folder. This can contain Helper classes and other utility functions.

/Runtime

1. The code should be written such that the outputs are written in the '/Runtime' directory. Before the submission of the repository, the files written in this directory should be moved to the '/SampleData/Output' directory for given inputs in the '/SampleData/Input' directory. The '/Runtime' directory will then be used to write outputs that would be written after a test run during the submission of model.

/Intermediate

1. This directory shall be used to write any intermediate outputs from the pre-processing of the model.

/main.py

1. The main python script (inference script) which will have the code to inference the Model present in /Models folder. The pre-processing, post-processing steps can be invoked from this script itself with their code either residing in '/main.py' or in any helper function present in separate file in '/Utils' Folder

/requirements.txt

1. List of python library requirements, we recommend generating this in a specific python virtual environment so that the global modules/library of python doesn’t affect the version or list of libraries required in case it is being generated via some third party tool like `pipreqs.

/README.md

1. General readme file regarding the Model Input/Model Output and other required information

/system_specs.txt

1. Any hardware/GPU related info/points requirements can be mentioned in this text file.

Appendix

Sl. no

Output Type

Expected Output File Type

Output File Sample

Image - Integer

TIFF File

An image file with different bands