Jupyter Notebooks

Interactive notebooks for learning and experimenting with LeRobot

LeRobot provides interactive Jupyter notebooks to help you learn robot learning concepts and experiment with the library in a hands-on way.

Getting Started with Notebooks

Installation

To use Jupyter notebooks with LeRobot, install the notebook dependencies:

pip install jupyter notebook
# or if using conda
conda install jupyter notebook

Launching Jupyter

From your LeRobot environment:

jupyter notebook

This will open a browser window where you can create and run notebooks.

Available Notebooks

While LeRobot focuses on Python scripts for reproducibility, you can easily convert any example into a notebook.

Creating Your Own Notebooks

You can create notebooks for any LeRobot workflow:

1. Dataset Exploration

   Load and visualize datasets interactively:

   from lerobot.datasets.lerobot_dataset import LeRobotDataset
   import matplotlib.pyplot as plt
   
   # Load dataset
   dataset = LeRobotDataset("lerobot/aloha_mobile_cabinet")
   
   # Visualize frames
   frame = dataset[0]
   plt.imshow(frame['observation.images.top'])
   plt.show()
   
2. Policy Training

   Train policies with interactive progress tracking:

   from lerobot.policies.diffusion.modeling_diffusion import DiffusionPolicy
   from tqdm.notebook import tqdm
   
   # Training loop with notebook-friendly progress bar
   for step in tqdm(range(training_steps)):
       loss = train_step(policy, batch)
       # Log metrics
   
3. Visualization

   Create interactive visualizations of robot trajectories and policies:

   import plotly.graph_objects as go
   
   # Plot robot trajectories
   fig = go.Figure(data=go.Scatter3d(
       x=positions[:, 0],
       y=positions[:, 1],
       z=positions[:, 2],
       mode='lines'
   ))
   fig.show()
   

Interactive Learning Resources

Robot Learning Tutorial

A free, hands-on course to learn robot learning using LeRobot

Hugging Face Datasets

Learn about the datasets library used by LeRobot

PyTorch Tutorials

Deepen your understanding of PyTorch, the foundation of LeRobot

Visualize on Hub

Explore and visualize datasets directly on the Hugging Face Hub

Common Notebook Patterns

Pattern 1: Dataset Inspection

from lerobot.datasets.lerobot_dataset import LeRobotDataset
from pprint import pprint

# Load dataset
dataset = LeRobotDataset("lerobot/pusht")

# Inspect metadata
print(f"Total frames: {len(dataset)}")
print(f"Number of episodes: {dataset.num_episodes}")
print(f"\nDataset info:")
pprint(dataset.meta.info)

# Visualize statistics
import pandas as pd
stats_df = pd.DataFrame(dataset.meta.stats)
stats_df.plot(kind='bar', title='Dataset Statistics')

Pattern 2: Policy Inference

from lerobot.policies.factory import make_policy
import torch

# Load pretrained policy
policy = make_policy(
    pretrained="lerobot/act_aloha_sim_transfer_cube_human"
)

# Run inference
with torch.no_grad():
    observation = get_observation()  # Your observation
    action = policy.select_action(observation)
    
print(f"Predicted action: {action}")

Pattern 3: Training Visualization

from IPython.display import clear_output
import matplotlib.pyplot as plt

losses = []

for step in range(num_steps):
    loss = train_step(policy, batch)
    losses.append(loss)
    
    # Update plot every 10 steps
    if step % 10 == 0:
        clear_output(wait=True)
        plt.plot(losses)
        plt.xlabel('Step')
        plt.ylabel('Loss')
        plt.title('Training Progress')
        plt.show()

Notebook Best Practices

Use Cell Magic for Timing

Measure execution time of cells:

%%time
dataset = LeRobotDataset("lerobot/aloha_mobile_cabinet")

Enable Auto-Reload

Automatically reload modules when they change:

%load_ext autoreload
%autoreload 2

Save Outputs

Save figures and models periodically:

# Save figure
plt.savefig('training_curve.png')

# Save model checkpoint
torch.save(policy.state_dict(), 'checkpoint.pt')

Memory Management

Clear GPU memory when needed:

import torch
torch.cuda.empty_cache()

# Or delete large objects
del large_dataset
import gc
gc.collect()

Converting Scripts to Notebooks

You can convert any LeRobot example script to a notebook:

# Install jupytext
pip install jupytext

# Convert Python script to notebook
jupytext --to notebook examples/dataset/load_lerobot_dataset.py

This creates a .ipynb file that you can open in Jupyter.

Cloud Notebook Platforms

Run LeRobot notebooks on cloud platforms:

• Google Colab: Free GPU access for experimentation
• Kaggle Notebooks: Free TPU/GPU resources
• Amazon SageMaker: Scalable cloud notebooks
• Paperspace Gradient: GPU-powered notebooks

Example Colab Setup

# Install LeRobot in Colab
!pip install lerobot

# Import and verify
import lerobot
print(f"LeRobot version: {lerobot.__version__}")

Sharing Your Notebooks

1. Clean Your Notebook

   Remove outputs and sensitive data before sharing

2. Add Documentation

   Include markdown cells explaining your approach

3. Share on Hub

   Upload to Hugging Face Spaces or GitHub

Share your notebooks with the community on Discord or GitHub Discussions!