TL;DR

We’re training models that can reason through physical tasks using chain-of-thought to automate traditionally difficult work in industrial environments like warehouses and manufacturing facilities.

How it works

We fine-tune vision-language models to output low-level robotic controls across primary tasks in logistics: mixed-sku palletization, item picking, sorting, and packing.

Before predicting the next action, our model is explicitly trained to <think></think> about its next step. This allows it to reason through long-horizon tasks.

https://youtu.be/axgbhBkkuQQ

RL from Verifiable Rewards

Our fine-tuned model is then rolled out in Nvidia’s Isaac Sim where it generates reasoning + action traces that get scored using simple verifiers. This continues to improve the model using photorealistic data without any human in the loop.

Our ask

We see logistics as an entry point to even greater automation across industries. Soon you will be able to tell AI to do any physical tasks in the same way current systems can handle the cognitive ones.

If you have any tasks you’d like to automate or know of anyone interested in working on embodied AI, please reach out: joshua@generaltrajectory.com

The Team

Joshua studied CS & neuroscience at the University of Chicago but spent most of his time outside of class doing ML research. He first joined NASA Ames Research Center and later worked at Caltech. His research focused on generally capable AI agents and reasoning.