Manifold Research Group tackles ambitious, high-impact research problems that traditional institutions overlook—those too engineering-intensive for academia and too exploratory for industry. Inspired by coordinated research models like ARPAs and FROs, we coordinate focused, cross-functional teams and a large asynchronous research contributor pool to systematically pursue and deliver paradigm-shifting science and technology.
MultiNet v2.0: Cross-Domain Multimodal Benchmarking
MultiNet v2.0 is a next-generation benchmark designed to test whether multimodal models truly understand tasks across fundamentally different action domains, or simply overfit to specific interfaces.
We construct puzzle environments where agents must perceive, reason about causal structure, plan, and act. Each task is presented across multiple action domains, including discrete grid environments, 3D physics simulations, and natural language interfaces. The task semantics remain identical, while the interface changes.
This allows us to directly measure whether models generalize across action spaces. Success across domains indicates real understanding; failure reveals interface-specific overfitting.
The Role
OS Team members form the core of Manifold Research Group. As an OS Research Fellow, you will contribute to building the benchmark platform and release artifacts for MultiNet v2.0.
In this role, you will be responsible for:
- Developing user-facing artifacts such as notebooks, websites, documentation, and example pipelines
- Building the benchmarking platform for running, visualizing, and analyzing results
- Creating clean interfaces and tooling for external users to interact with MultiNet environments
- Supporting packaging and release of datasets, environments, and evaluation tools
- Collaborating with researchers to translate experimental systems into usable, well-documented artifacts
Qualifications
Outstanding research emerges from individuals who can translate complex systems into clean, usable tools. For this role, we are looking for:
- Strong software engineering skills with attention to usability and system design
- Experience building developer tools, APIs, or research-facing platforms
- Familiarity with Python and backend or lightweight frontend tooling
- Ability to write clear documentation and design intuitive user workflows
- Experience contributing to open-source projects is preferred
Expectations
There are a few key expectations and clarifications regarding the OS Research Team:
- Contribute approximately 10 hours per week to ensure meaningful progress and deep engagement with our projects. Flexibility around life commitments is understood; clear, proactive communication helps us support each other.
- Our working language is English, and strong proficiency is required to clearly communicate technical concepts without confusion or misunderstanding.
- This is a volunteer effort; none of us receive compensation of any kind—including monetary payment, academic credit, or other formal incentives. Our commitment is driven entirely by shared passion for impactful research.
More information on OS Research Team expectations is available here.
We look forward to seeing your application, and hopefully working together soon!