Submit Your Model

Join the 2025 SCT-Bench Competition

Submission Process

To participate in the SCT-Bench competition, please follow these steps carefully:

1. Registration

Email your registration request to sctbench@gmail.com with the following information:

  • Team Name
  • Institution/Organization
  • Lead Researcher Name
  • Professional Email Address
  • Brief description of your model (2-3 sentences)
  • Link to your team's research page or GitHub repository (if applicable)

Note: We review all applications to ensure responsible use of the benchmark.

2. Technical Requirements

Computation Limits

  • Maximum 4 CPU cores
  • Maximum 32GB RAM
  • Maximum 2x 80GB A100 GPU
  • Maximum runtime: 1 hour

3. Submission Requirements

Your submission package must include:

  • Python script for model evaluation (based on our starter code)
  • requirements.txt file (pip only, no conda)
  • .env file (if using third-party APIs)

Required Starter Code

All submissions should use our starter code as a base to ensure consistent evaluation. The starter code:

  • Provides the standard input/output format required for evaluation
  • Includes example predictions on the training set
  • Handles proper logging and error reporting

Download the starter code from our GitHub repository and refer to the included main.py for the required output format.

Important Notes

  • Please use our training set to improve your model and verify compatibility with the benchmark
  • Teams are limited to 1 submission per day
  • If using third-party APIs, include all necessary API keys in an .env file
  • Ensure your code can handle API rate limits and temporary outages

4. Code Structure

Your submission should follow this basic structure:

project/
├── main.py           # Main evaluation script
├── requirements.txt  # Python dependencies
├── .env             # API keys (if needed)
└── README.md        # Setup instructions

5. Evaluation Process

  1. Submit your package to sctbench@gmail.com
  2. We will verify your submission meets all requirements
  3. Your model will be evaluated on our private test set
  4. Results will be added to the leaderboard within 5 business days

Fair Use Policy

By submitting to SCT-Bench, you agree to use the benchmark responsibly and not attempt to reverse engineer the test set or game the evaluation system. Violations will result in immediate disqualification.