Cosmos-Tokenizer-CI8x8-Lidar: A tokenizer for Lidar range map

Model overview

Description:

Cosmos-Tokenizer-CI8x8-Lidar tokenizes LiDAR point cloud data into continuous latent representations for efficient compression and generation tasks. This tokenizer encodes and decodes LiDAR range map data, providing high-quality reconstructions with significant improvements in reconstruction accuracy after fine-tuning on Lidar range map data. Lidar tokenizer was developed by NVIDIA as a part of the Cosmos Tokenizer family.

This model is ready for commercial/non-commercial use.

License/Terms of Use

This model is released under the NVIDIA Open Model License. For a custom license, please contact cosmos-license@nvidia.com.

Under the NVIDIA Open Model License, NVIDIA confirms:

• Models are commercially usable.
• You are free to create and distribute Derivative Models.
• NVIDIA does not claim ownership to any outputs generated using the Models or Derivative Models.

Important Note: If You bypass, disable, reduce the efficacy of, or circumvent any technical limitation, safety guardrail or associated safety guardrail hyperparameter, encryption, security, digital rights management, or authentication mechanism (collectively “Guardrail”) contained in the Model without a substantially similar Guardrail appropriate for your use case, your rights under this Agreement NVIDIA Open Model License Agreement will automatically terminate.

Deployment Geography:

Global

Use Case:

Research related to autonomous driving; enabling efficient compression and reconstruction of Lidar data for generative AI models and LiDAR processing pipelines.

Release Date:

Hugging Face [11/03/2025] via [https://huggingface.co/nvidia/Cosmos-Tokenizer-CI8x8-Lidar]

References(s):

Cosmos-Dream-Drive

Model Architecture:

Architecture Type: Auto-Encoder

Number of model parameters: 7.7*10^7

Network Architecture: This model was developed based on Cosmos-0.1-Tokenizer-CI8x8. We designed Cosmos Tokenizer using a lightweight and computationally efficient architecture, featuring a temporally causal design. Specifically, we employ causal temporal convolution and causal temporal attention layers to preserve the natural temporal order of video frames, ensuring seamless tokenization of images and videos using a single unified network architecture. The encoder and decoder form a symmetrical pair, which are mirrors of each other. The encoder starts with a 2-level Haar wavelet transform layer, which down-samples inputs by a factor of 4 in both spatial and temporal dimensions. Likewise, the decoder ends with an inverse wavelet transform. We employ the vanilla autoencoder (AE) formulation to model the latent space for continuous tokenizers.

Input:

Input Type(s): Lidar data
Input Format: Lidar range map, 1 x H x W
Input Parameters: Two-Dimensional (2D)
Other Properties Related to Input: LiDAR point cloud data with range coverage of 5-100 meters, the original resolution of the range map is 128 x 3600. We repeat rows by a factor of 4 and downsample the columns by a factor of 2, the final processed range map is of resolution 512 x 1800. The spatial compression rate is 8x8. This tokenizer is compatible with Nvidia internal 128-beam LiDAR sensor only. Users can post-train their own lidar tokenizer following the provided guidance.

Output:

Output Type(s): Lidar
Output Format: Lidar range map, 1 x H x W
Output Parameters: Two-Dimensional (2D)
Other Properties Related to Output: Continuous latent space

Software Integration:

Runtime Engine(s):

• Cosmos-Drive-Dreams

Supported Hardware Microarchitecture Compatibility:

• NVIDIA Ampere
  
• NVIDIA Blackwell
  
• NVIDIA Hopper
  

Preferred/Supported Operating System(s):

• Linux
  

The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.

Usage

• See Cosmos-Drive-Dreams for inference details.

Note: We have only tested doing inference with fp32 precision. On a single NVIDIA A100-SXM4-80GB GPU, our tokenizer runs at 1.7 fps for a processed lidar range map of shape 512 x 1800.

Model Version(s):

Cosmos-Tokenizer-CI8x8-Lidar

Training, Testing, and Evaluation Datasets:

Data Collection Method:

• AV: Automatic/Sensors

Labeling Method:

• AV: Hybrid: Human, Automated

Inference:

• Acceleration Engine: None
  
• Test Hardware: NVIDIA NVIDIA A100-SXM4-80GB

Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please make sure you have proper rights and permissions for all input image and video content; if image or video includes people, personal health information, or intellectual property, the image or video generated will not blur or maintain proportions of image subjects included.

For more detailed information on ethical considerations for this model, please see the subcards of Explainability, Bias, Safety & Security, and Privacy below. Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns here.

Bias

Field	Response
Participation considerations from adversely impacted groups protected classes in model design and testing:	None
Measures taken to mitigate against unwanted bias:	None

Explainability

Field	Response
Intended Task/Domain:	LiDAR Point Cloud Tokenization and Reconstruction
Model Type:	Auto-Encoder
Intended Users:	Researchers and developers working with autonomous driving, robotics, and 3D perception systems.
Output:	Continuous latent representations of LiDAR range maps for efficient compression and reconstruction
Describe how the model works:	LiDAR range map data is encoded into continuous latent tokens through a temporally causal auto-encoder. The encoder uses 2-level Haar wavelet transform for spatial downsampling, followed by causal temporal convolution and attention layers. The decoder mirrors this process to reconstruct the original LiDAR range maps with 8x8 spatial compression.
Name the adversely impacted groups this has been tested to deliver comparable outcomes regardless of:	Not Applicable
Technical Limitations & Mitigation:	This model is compatible with NVIDIA internal 128-beam LiDAR sensor only. Performance may degrade with different LiDAR configurations, extremely sparse point clouds, or data outside the 5-100 meter range coverage. Users should validate model performance on their specific LiDAR sensor setup and environmental conditions.
Verified to have met prescribed NVIDIA quality standards:	Yes
Performance Metrics:	Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Relative Error
Potential Known Risks:	If the model does not work as intended, it may produce inaccurate LiDAR reconstructions that could lead to incorrect 3D spatial understanding in autonomous systems. This could result in navigation errors, collision risks, or poor object detection performance in safety-critical applications.
Licensing:	NVIDIA Open Model License

Privacy

Field	Response
Generatable or reverse engineerable personal data?	No
Personal data used to create this model?	No
How often is dataset reviewed?	Before Release
Is there provenance for all datasets used in training?	Yes
Does data labeling (annotation, metadata) comply with privacy laws?	Yes
Is data compliant with data subject requests for data correction or removal, if such a request was made?	Yes
Applicable Privacy Policy	https://www.nvidia.com/en-us/about-nvidia/privacy-policy/

Safety

Field	Response
Model Application Field(s):	Autonomous Vehicles, Industrial/Machinery and Robotics
Describe the life critical impact (if present).	Could be incorporated into autonomous machines. As a tool for generating synthetic environments, caution should be used for testing AI models based on data generated from this model.
Use Case Restrictions:	NVIDIA Open Model License
Model and dataset restrictions:	The Principle of least privilege (PoLP) is applied limiting access for dataset generation and model development. Restrictions enforce dataset access during training, and dataset license constraints adhered to.