Idea

May the force...

Some projects begin with a problem; this one began with curiosity and a love of Star Wars. Few scenes capture the imagination like Luke Skywalker's training aboard the Millennium Falcon—deflecting laser bolts from a hovering remote drone with his lightsaber while blindfolded. That image of playful, physical skill-training stuck with me and raised a question: what is actually possible with computer vision today?

Marksman-H is the answer to that question—a small game built in Unity3D that experiments with using a regular camera as a real-world controller for a virtual game. Rather than solving a defined problem, the project is an exploration of technology's capabilities: Can a simple Python script track a physical object accurately enough to feel responsive? Can a $10 prop and a webcam deliver an experience that dedicated controllers promise? Framed as an experiment, the stakes were refreshingly low and the freedom correspondingly high—whatever worked became the game, and whatever delighted me stayed in.

Implementation

Turning a stick into lightsaber

At the heart of Marksman-H is a simple prop: a stick with a tennis ball attached to each end. This design choice was deliberate—the two brightly colored balls give the computer vision system distinctive, easily trackable reference points at both ends of the implement.

A Python script using OpenCV processes the camera feed to detect the positions of both balls in the frame. From these two points, the stick's orientation can be calculated—the angle, tilt, and movement direction are derived geometrically. This orientation data is then translated into the game world, where a virtual lightsaber mirrors the physical stick's motion in real time. Swing the prop, and the saber follows; rotate it, and the blade rotates with it.

The gameplay itself pays homage to the scene that inspired the project: laser beams are fired toward the player, and the goal is to deflect as many as possible with the virtual lightsaber. The game loop is immediate and physical—the player ducks, twists, and swings, just as Luke did. What appears on screen is a duel, but what happens in the room is a workout.

Experience & Learnings

Much to learn, I still have...

What makes Marksman-H memorable is the moment it clicks: the disappearance of the interface. Players stop thinking about the stick, the camera, or the script—there is simply a lightsaber in their hand. Latency becomes the decisive factor in this illusion; the vision pipeline had to be lean enough that reaction felt instant rather than observed. Tuning detection thresholds, ball colors, and lighting conditions consumed much of the development time, underscoring a fundamental truth of markerless computer vision: robustness lives in the details.

As an experiment, the project delivered a clear takeaway—accessible, open-source tools like OpenCV paired with a standard webcam are sufficient to create compelling whole-body interaction, no special hardware required. And as a personal piece, it proved something simpler: that the training drone from a galaxy far, far away is, technologically speaking, not so far away after all.