Work that takes two, recorded from both heads.
Duet collects video of two people doing one job together, each wearing a head camera. The footage teaches robots to work alongside people. This guide is for sites in Nepal that run those recordings: what to record, what a session must contain, how we check it, and what it pays.
The same twelve seconds from both head cameras: A passes the vase to B, then starts arranging the flowers.
Same clock 0:48- Worker pay
- NPR 200 for each worker, per hour of usable footage
- Site fee
- Negotiable. Agree it with your Duet contact in Nepal.
- You deliver
- Two head-camera videos per session, raw and uncut. Duet does the labelling.
A real recording, 32 seconds
Two people unpack a makeup organizer and sort what is in it. B takes items out and hands them across the table; A lays them out. Both head cameras show the same 32 seconds at real speed. Nothing is edited or labelled.
Two people, not two hands
Bimanual data follows one person's two hands, and most public first-person datasets also follow one person. We record two people doing one job, with a camera on each head, because what a robot has to learn is how each person reacts to the other.
- Bimanual data
- One person, two hands, one camera view.
- What we record
- Two people, four hands, two head cameras on one clock. Each camera also sees the partner.
Any work two people do together
If a job has two people working with each other, it counts. Here are three common kinds.
-
Handovers
Passing a partner the tool, part or item they need while their hands are busy.
Parts at a bench, items for a box, tape, scissors
-
Four hands on one thing
One holds or steadies it while the other works, or both fold, wrap and shape it together.
Packing, taping, assembling, folding, wrapping
-
Too big for one
Lifting, carrying, turning and setting down something one person cannot control alone.
Tables, mattresses, cupboards, rolled carpets, long panels
Any other work counts too, as long as two people do it together.
Side by side, each on their own
Two people doing separate tasks in the same room. Nothing passes between them, so there is nothing here to learn about working together.
Whole jobs, where they already happen
One carry takes forty seconds. Rearranging a room takes most of an hour and has twenty carries in it, each with a different route, grip and turn. Record the whole job, including the pause to work out where the table goes. These workplaces already have such jobs.
- Packing rooms
- Sorting and packing at a table: one holds the container while the other fills it, items passed across.
- Assembly workshops
- One holds the parts in place while the other fastens them.
- Carpet and handicraft exporters
- Carrying rolled carpets, building a carton around a long item, one holding while the other tapes.
- Hotels and guesthouses
- Making up a room: flipping a mattress, folding sheets, getting a quilt into its cover.
- Party palaces and event crews
- Folding tables, tablecloths and a banner set up, then all of it packed away again.
- Movers and furniture workshops
- A sofa, a table and a cupboard to new places, through a doorway and back.
What a session must contain
One session is one job, recorded from both heads at once.
- Two videos per session
- One from each person's head camera, started together and covering the whole job.
- First-person view
- The hat's camera looks where the hands work. Each person's hands are in view for more than 85% of the session.
- Two claps to line them up
- At the start and again at the end, one person claps twice with hands held high, where both cameras can see them. We use the claps to put the two videos on one clock.
- Uncut and unlabelled
- No trimming, no editing, no labels. The cameras keep running between tasks. Duet does the labelling.
- A row in the session log
- The date, the place, the job, and who wore which camera.
- The example's format
- Stereo head camera, 1280 × 720 per eye, 30 frames per second, shrunk for the web.
- Camera
- Duet sends custom camera hats and pays for all the hardware.
- Session length
- About 45 minutes: one job, start to finish. A pair can record several sessions a day.
One session, start to finish
The same routine every session.
- 01
Before
Room lights are on full and stay that way. The floor is clear along the route. Each camera sees its wearer's hands at chest height.
- 02
Start
Start both cameras. One person claps twice, about a second apart, with hands held high where both cameras see them.
- 03
The job
The pair does the whole job without stopping the cameras. If something is dropped, they recover on camera. Stop only if someone is at risk.
- 04
End
Clap twice again, the same way, then stop both cameras.
- 05
Log
Fill in the session log the same day, while it is fresh.
- 06
Upload
At the end of each day the site manager collects the hats and uploads that day's footage.
What to tell the people you record
They should work the way they always do. These are the points to pass on. The Nepali version of this page has the same list in Nepali.
Do
- Do the whole jobStart to finish. Planning, pauses and changes of mind are part of the job.
- Keep your hands in viewThe head camera should see your hands whenever you hold something in front of you.
- Leave the mistakes inIf something slips or will not fit through the door, recover on camera. Those moments are rare and worth a lot.
- Swap roles if the job allowsHalfway through, change who leads and which side each of you stands on.
- Use the real objectsReal things at their real weight, with no lighter stand-ins.
Don't
- Solo work side by sideIf each of you could leave the room without the other noticing, it does not count.
- Acting for the cameraDo not turn a one-person job into a two-person job, and do not slow down.
- PhonesIf one of you is waiting, stay ready and watch your partner.
- Anything unsafeOnly lifts you would do without the camera. No stairs, nothing overhead, nothing heavy enough that a slip would hurt.
Different people, places and jobs
The same number of hours is worth more when it is spread across people, places and jobs.
Many pairs
Rotate who works with whom. Mix heights, left- and right-handed people, and pairs who know each other with pairs who do not.
Several places
Use more than one room or work area, and vary the lighting from one session to the next, never during one.
Changing jobs
Rotate the jobs, and change the route, the grip and who leads between sessions.
Things going wrong
A grip that slips, a doorway that will not fit, the wrong item handed over. Keep these, and note the time in the session log.
What makes a session count
We check every session against the same list. One that fails is recorded again, not repaired.
The two videos line up
Both claps are visible in both videos, so the two can be put on one clock.
Hands are in view
Each wearer's hands are visible for more than 85% of the session.
The pair works together
There is joint work or a handover at least once every two minutes.
Nothing is cut
Each video runs from the first clap to the last without a break.
The log is filled in
The session has its row in the session log.
Two views of the same moment
Each camera sees its wearer's hands and the partner across from them. Together the two views show where both people and all four hands are, and who is doing what.
- Video on one clock
- Both head cameras, lined up on the same clock.
- Heads and hands in 3D
- Where each person's head and hands are, in one shared space measured in meters.
- Steps and handovers
- What each person is doing, when they work together, and every time something changes hands.
Working togetherWaitingHandover · vase · A to B · 0:54
From one real 10-minute session with 19 handovers. Labels are made automatically from both videos. See the full session →
What it pays
- Workers
- NPR 200 for each worker, per hour of usable footage
- The site
- Negotiable. Agree it with your Duet contact in Nepal.
- Site manager
- One person at each site is paid extra as site manager, for collecting the hats and uploading the footage every day. The amount is negotiable.
- Payment
- Usable means the session passes the checks in this guide. How and when payment is made is agreed with your Duet contact in Nepal.
Email Duet
Email us the points listed here. The person who sent you this guide works with Duet in Nepal, and you agree your fee with them.
andrew@duetlabs.co- Where
- The city or district in Nepal.
- The work
- What two-person work happens there, or what you could set up.
- Capacity
- How many pairs you can record, and how many hours a week.
- Your fee
- What you would charge for running the recordings.