Sensorimotor Exploration Lab: Handwriting Robot for Experimentation and Physical Rehabilitation

I worked part-time at the Sensorimotor Exploration Lab during my junior year. The lab gave me the opportunity to program a Kinova Gen 3 Ultralightweight Arm to write words. The system supported a research experiment that compared four methods of learning new words: viewing a static word, watching it being written, viewing and then writing it, and being physically guided through the motion by a robot. The follow-up test showed the best memory performance when participants viewed and then wrote the word. Robot-guided writing placed second, followed by watching the word being written and viewing it statically. These results suggest that physically performing a writing motion can improve learning.

Technical Challenges

1. Manipulator Controls

My first task was to resolve issues in a previous engineering capstone project. Their implementation produced visible jitter while writing, making the trajectories unsuitable for controlled experiments. The vibration is visible as the end effector moves vertically in the video below.

I traced the problem to the inverse-kinematics algorithm in Kinova’s built-in C++ API. The solver is useful for fast, approximate IK lookups but is not designed as a high-performance, numerically optimal trajectory controller. Running it in real time on the robot’s onboard CPU introduced discontinuities and noise.

Encoder data showed noisy kinematic profiles that exceeded the intended limits.

The next challenge was writing a continuous inverse-kinematics algorithm that could generate trajectories from Cartesian coordinates.

I used a kinematics library from Professor Matt Pan’s work at Disney Research and implemented a pseudoinverse-Jacobian method to generate smooth trajectories before runtime. The algorithm takes the current joint angles, uses forward kinematics to calculate the current pose, measures the Cartesian error relative to the target, and uses the Jacobian to convert that error into a joint-space correction. It repeats this process until the Euclidean error is sufficiently small.

In simple terms, the algorithm repeatedly takes an educated guess, evaluates the result, and corrects itself until it reaches the target.

X is subtracted from Xd until e is negligible, indicating that q has converged.

I implemented the method over the holiday break without access to the physical robot, so I verified it in simulation. I compared the inverse-kinematics results from MATLAB Simulink’s built-in tool with my C++ implementation. Matching Cartesian outputs for the same joint inputs confirmed that the solver was working.

I obtained a CAD model of the Kinova arm and followed this tutorial to export it from SolidWorks into MATLAB Simulink.

MATLAB converts the CAD assembly into a usable simulator. It generates the linkage transforms from the SolidWorks mates, accepts joint-angle inputs, and returns Cartesian coordinates.

Kinematic model generated by MATLAB from SolidWorks mates.

In red on the left I input joint positions (degrees) to the kinematic model (blue) and I get cartesian coordinates (green) on the right

Simulink also provides a 3D representation of the manipulator pose for each set of joint inputs.

After calibrating the simulated home position, I passed target poses into my inverse-kinematics algorithm and returned the solved joint angles to Simulink. Reconstructing the original target pose provided an end-to-end check of the implementation.

I then developed a full trajectory generator from an array of Cartesian positions. A CODAMotion camera supplied timestamped positions at 400 Hz. The function imports the points from CSV, aligns their starting position, adds vertical motion for contacting and leaving the writing surface, and performs IK while handling singularities, non-invertible Jacobians, and non-convergence. It then interpolates the 400 Hz data for the Kinova controller’s 1 kHz update rate, applies joint-velocity and acceleration limits, and formats the resulting commands for the robot.

Velocity and acceleration limiting allowed me to send asynchronous position commands at each 1 ms time step. The controller relied on Kinova’s onboard actuator loops to track each command before the next update. Testing confirmed that any tracking delay was small enough to produce smooth results.

The final step was making the code accessible to researchers without a programming background. I exposed parameters through a JSON file so users could select trajectories, speeds, and other experiment settings without modifying the source code.

2. End Effector

I redesigned the end effector to be more ergonomic and maintain consistent contact with the writing surface. The SolidWorks design allows the spring-loaded pen to move along its axis, maintaining contact when the robot presses into a table or whiteboard.

Learnings & Takeaways

This project developed my experience with cobots, trajectory control, and experimental research. I am grateful to Dr. Manson, Dr. Pan, and Obi for the opportunity and support.

Union and University and the Kin building where I worked (right)