A Multi-Robot AI for Jogging Street Art
One of the challenges in the fitness domain is to gamify fitness – to make it fun, exciting, and engaging so that people are motivated. Often times, the best form of encouragement occurs in groups where teams of multiple people play some kind of game or take part in some team challenge to collectively improve or meet their fitness objectives.
Multi-robot AI planning is concerned with creating algorithms to control and plan routes for swarms of robots, agents, cyborgs, etc where often the robotic agents are collaborating to accomplish some team objective. Multi-robot systems, thus, provides an algorithmic framework to help optimize both individual and team objectives in fitness.
In this study, we propose CyFitNet, a possible application that builds upon a popular meme on Facebook involving jogging street art. Using apps such as RunKeeper or MapMyRun, users create drawings and artwork with the geographic trajectories they jog in the real world. Users then post their runs/drawings for others to see (and possibly compete against).While most of the time these runs/drawings are made by individuals, we believe it is possible for a medium to large-scale team of joggers to run/draw a larger image such as a text message or a famous work of art. We believe that when individuals are presented with an individual objective (such as drawing one part of an artwork) that is part of a larger team objective (i.e. drawing the entire artwork), they are more likely to do their part.
To facilitate this objective, we develop a prototype system that allows users to upload any image that they wish to draw while running. The system decomposes the overall image as a set of trajectories. Extracted trajectories are mapped onto the real world (in a geographic space selected by the user) so that a team of joggers could draw the image while running the trajectories. Finally, multi-robot algorithms are used to assign trajectories (from the full set of trajectories) to a team of joggers to create the image. The algorithms flexibly take into account various preferences of the individual joggers such as where they are starting their run, fitness objectives such as what total distance they want to run, and associated weights for preferences while meeting the team objective of completing the picture.
II. System/Algorithm Design
The underlying system has several algorithmic steps that enable function.
Step 1: The user uploads an image to the server that contains a drawing or message they wish to jog. We will use as our example an image of coins (shown in Figure 1).
Step 2: A Canny Edge Detection algorithm is run to produce an edge map of the image. An edge map classifies each pixel in the image as being the location of an edge or not. The edge map can help identify edge pixels of the underlying drawing (results shown in Figure 2).
Step 3: An Edge Extraction algorithm is run to extract continuous edges from the edge map. These extracted edges form a candidate set of trajectories that a team of joggers will run to draw out the image. Extracted trajectories/edges are shown in Figure 3.
Step 4: The extracted trajectories/edges are mapped onto a geographic region in the real world. The trajectories are transformed into Latitude / Longitude space and transferred to a geographic location specified by the user. For our example, we map the trajectories to streets near Monta Vista High School and plot on Google Maps (see Figure 4).
Step 5: Once the drawing is specified, the locations of our jogging team are queried along with their preferences. In our example, we simulate N=20 joggers in the region ready to run our routes, along with individual jogger fitness objectives, such as the total distance each jogger wants to run today. The locations of the joggers, along with the set of possible trajectories, are plotted in Figure 5.
Step 6: A multi-robot planning algorithm analyzes the preferences of the joggers as well as the possible trajectories to be run and comes up with an optimal assignment of the trajectories to the joggers. The underlying multi-robot solver algorithm is based on the binary integer programming solution to the Assignment Problem ( in Multi-Agent Active Learning (Tandon 2012). We assume that the cost of the overall team assignment is decomposable as the costs to the individual agents. The fitness function used takes into account the following factors:
- The distance between the starting position of the jogger to the start of the trajectory
- Whether the total distance traveled by the jogger is within the fitness objective specified by the jogger (i.e. If the user says they want to jog a minimum of 1 mile and maximum of 2 miles, then that constraint is taken into account in the route assignment).
- Whether as much of the team picture is completed as much as possible.
All these constraints can be plugged into the Assignment Problem (http://en.wikipedia.org/wiki/Assignment_problem). bintprog in MATLAB was used to solve the binary integer program (https://lost-contact.mit.edu/afs/cs.stanford.edu/package/matlab-r2009b/matlab/r2009b/toolbox/optim/optim/bintprog.m).
In terms of the formulation, the cost C can be made of constraint #1 and constraint #3 can be plugged into f while consideration #2 can be plugged into b so you get an optimal solution from the solver without doing much else.
Figure 6 shows an example optimal assignment of routes to the joggers taking into account these constraints using these costs in our fitness function optimization.
The end result is a trajectory that each jogger can run that satisfies their fitness objectives, maximizes according to their preferences for routes, while maximizing the overall team objective of completing the picture.
In this study, we prototype a possible useful multi-robot application to the fitness domain. Our system helps gamify jogging via jogging in teams to create jogging street art. The developed algorithms automatically extract trajectories from an uploaded artwork or message, assign them to a team of joggers to draw out the picture while running, all the while taking into account each individual joggers’ unique fitness objectives. We hope our system will encourage people to jog more and create beautifully stunning works of art.
Ongoing challenges are to improve the multi-cyborg AI:
1. Mapping to actual streets may prove a challenge for non-rectangular, non-square images. The application may work better in areas in the world with plenty of roundabouts for optimal angular directions and seemingly random projections of the data onto the world.
2. One of the major challenges with controlling robotic-humans (i.e. cyborgs) as opposed to just robots is that robots typically follow programmed directions. In contrast, cyborgs have two controllers (i.e. the original mind as well as the computer controller) so may choose to override directions.This happened with some of our user tests. Some of the joggers chose not to follow the paths given to them and drew human reproductive parts and obscenities instead. The challenge thus is to spot cyborgs that are running uninstructed paths from messing up the picture. We hope to develop appropriate error correction and detection mechanisms so that algorithms can be robust to cyborgs that may corrupt the data. This double brain problem is, in general, a major research challenge with cyborg systems that does not come up as much in classical robotics.