This is one of those things that sounds really simple in theory but then ends up being incredibly complicated when you actually try to do it! The system that ESPN uses to paint the line is called "1st and Ten" and is created by a company called SporTVision.
The simplest description of this system is as follows: "Knowing where the first down line is (for example, at the 42 yard line), draw that line on the field with a computer so that viewers seeing the game on TV can see the line as though it were painted on the field." Here are some of the problems that have to be solved in order for this system to work:
There are probably several other glitches as well... It's a tough problem!
- The system has to know the orientation of the field with respect to the camera so that it can paint the first down line with the correct perspective from that camera's point of view.
- The system has to know, in that same perspective framework, exactly where every yard line is.
- Given that the cameraperson can move the camera, the system has to be able to sense the camera's movement (tilt, pan, zoom, focus) and understand the perspective change because of the movement.
- Given that the camera can pan while viewing the field, the system has to be able to recalculate the perspective at a rate of 30 frames per second as the camera moves.
- A football field is not flat - it crests very gently in the middle to help rainwater run off. So the line calculated by the system has to appropriately follow the curve of the field.
- A football game is shot by multiple cameras at different places in the stadium, so the system has to do all of this work for several cameras.
- The system has to be able to sense when players, referees or the ball cross over the first down line so it does not paint the line right over the top of them.
- The system has to also be aware of superimposed graphics that the network might overlay on the scene.
To solve these problems, the creators of the 1st and Ten system combine hardware and software. First, each camera must have a very sensitive encoder attached to it that can read the camera's angle, tilt, zoom and so on and send that information to the system. The system must also have a detailed 3-D model of the field so that it knows where each yard line is. By integrating the tilt, pan and zoom information with the 3-D model, the system can begin to calculate where the line should go. Then the system uses color palettes for the field and the players/referees/ball to recognize, pixel by pixel, whether it is looking at the field or something else. This way, only the field gets painted.
According to the SporTVision web site, all of this computation requires a lot of equipment: "There are eight computers (four SGIs and one PC and three special purpose data acquisition computers), three sets of special encoders and abundant wiring dedicated to generating the virtual first down line in video format." Who would have thought...
Here are two interesting links: