Naz Louis has upgraded his open-source robot Pixel, giving the walking desk robot a new AI system that can see, remember, and reason about its surroundings in real time. The new build runs on Gemini Robotics ER2, an embodied-reasoning model built by Google DeepMind for near real-time robotics applications. The model receives a continuous stream of camera and microphone input and has direct access to functions controlling Pixel's movement, expressions, and speech, along with data from an onboard IMU that tracks orientation and detects falls.
Louis also added a continuous vision pipeline that analyzes multiple frames per second instead of single static images. To give Pixel spatial awareness without a dedicated depth sensor, he integrated a separate model trained specifically to estimate distance from ordinary 2D camera images. Pixel's memory system carries over a set of text files, covering its identity, its understanding of Louis, and a running memory log, that the robot periodically rewrites based on daily interaction logs.
In testing, Louis put Pixel through a series of behavioral trials, including a mirror recognition test and a scenario in which Pixel walked toward a smaller companion robot near an open flame. Louis plans to release the code for this version, sharing it through Patreon posts that link to a public GitHub repository.
