Video Analysis Software Review
In this review we will look at eight software packages in terms of design, functionality and cost.
The use of video is ubiquitous today for both personal and professional applications. Video systems are becoming cheaper and small with higher resolution, increased storage capacity and lower power requirements. Due to this, cameras are a standard tool in any researcher’s toolbox. However the collection of video tends to be limited by the resources required to analyze those observations. While automated analysis still requires extensive research due to its complicated nature, particularly in the underwater environment, software tools are being developed to help researchers efficiently analysis video observations. The advantage of using a dedicated video analysis software package is that they typically maintain the linkage between the video file (input) and the data file (output). Having a video/data embedded file allows researchers to pick any data point and easily go back to the video observation where it was collected. This makes it easier to review, edit or collect more data from a particular observation.
The goal of this review was to look at different software packages readily available; both free and commercial products. An extensive online search was conducted looking at potential software options from a wide range of research fields (psychology, primatology, sports, physics, etc.). Software packages were initially examined to determine their applicability. Eight packages were obtained for further examination (Table 1). It was noticed that all software packages fell into one of two categories. Event coding tools allow the observers to document events, behaviors or states, typically from a pre-defined list. Analytical measurement tools allow the user to measure variables like angles, headings and distances observed in the video. The specifics of each category will be discussed in subsequent sections. Unfortunately there was no single package with both utilities. As such, this review only compares software packages within its respective category.
Each software was evaluated for its overall performance, functionality (options, customizations, etc.), learning curve for getting started, ease of use (graphic interface, layout, quick keys) and exporting/reporting functionalities.
All the event coding packages operate in roughly the same fashion. These methodologies have been developed through their use in other field of research, particularly psychology and ethology. While there is extensive literature on conducting behavioral studies, I will provide a brief overview of the fundamental design of these systems so all readers will have an adequate knowledge base for understanding the advantages and disadvantages of each system.
Typically all behavioral studies are based on the development of an initial ethogram, a list or catalog of behaviors or actions in which the observer plans to document. These ethograms can be extended to documents variables, conditions or states which might be useful in subsequent analysis. Ethogram objects can be listed as state or point events. State events typically have a start and stop function (i.e. fish is swimming). Point events are typically momentary events (i.e. fish escapes from a net). Modifiers can be added to the initial object. For example, swimming fish might be coded as burst swimming, sustained swimming or gliding. If individuals are being studied, this information may be coded per subject. Subject meta-data is typically also recorded. Additionally meta-data on independent variable (i.e. weather, sea state, etc.) can be documents for later data analysis. Finally, all coding is done per observation. An observation is the unit of time in which events are coded. These are typically pre-defined, such as an hour, minute, first ten minutes of every hour, feeding event or tow, and each typically serves as an experimental replicate.
In this review we looked at five different software packages. Of these, the most powerful and versatile was Observer XT 12 by Noldus. Observer had the most extensive set of options and modifications in which to customize the analysis. This was to be expected given the high price tag. With slightly less customizations then Observer, I really liked BORIS. It provided good functionality that would be needed in most studies yet was free.
File Formats Supported: It was hard to find an explicit list
Overall, Observer XT is a great piece of software for coding video events. With a product manual numbering 504 pages, start-up time can be a bit long, mainly due to the numerous options and features available. However getting started with basic analysis is probably around 1 hour. As previously stated, this software has numerous options/feature allowing the users to customize the set-up for their needs. Once observation coding is completed, Observer had a nice set of analysis, reporting and exporting tools allowing for quick processing, graphing or exporting of the data. The primary drawback of the system is the cost compared to similar free software options.
· Numerous features and options allow for a custom analysis designed to work for your study.
· In-depth documentation (500+ pg manual) and support from the company.
· Multiple video playback with synchronization options.
· Time offsets of videos. Typically, events are logged relative to the time position of the video, however providing an offset can allow for logging relative to clock time.
· Quick keys
· Event and state coding with modifiers
· Undockable modules allow the user to move the program field to fit their screen.
· Syncing with external data
· Event code modifiers can be numerical or nominal.
· Many data processing, analysis, reporting and exporting options.
· A lack of fine control over playback speeds (1x, 2x, 4x , etc). Playback at 2x and 4x was very choppy and would not be useful for analysis.
· No geometric analysis options.
· Due to the large number of options, set-up was a bit confusing.
File Formats Supported: All file formats supported by VLC
BORIS (Behavioral Observation Research Interactive Software) is a great new software package in early development for analyzing and coding observations from video. Video can be analyzed via a live feed or post-processed from a video file. Coupling with VLC provides a wide array of importable formats. The documentation provides a quick but effective utility to get up in running in around 30 minutes. Additionally this is a package which is in active development. I requested a feature on their website and it was quickly implemented in the latest version.
· Good documentation and easy set-up allows for start-up in under an hour.
· VLC support provides a wide array of import options.
· Event vs state coding
· Time offsets of videos.
· Dual video playback with synchronization
· Ability to load a playlist.
· Undockable menus provides customization options
· Quick keys
· Use of secondary modifiers or coding maps increases the information of each event or state.
· Coding maps can log spatial information about regions were events occurred.
· Export functions allow data to be exported as text file.
· BORIS has basic reporting functions such as providing a tally table of event counts or state durations. Data can be view in a basic timeline graph.
· Event codes can use either a secondary modifier or a coding map; however both cannot be used together.
· No capability to take geometric measurements.
Cowlog is a very simple, basic piece of software for documenting events from video files. The program can be run as a desktop application or from a web browser. Cowlog only supports event codes, however modifiers may be added. The interface is simple and unobtrusive allowing for more room for the video screens. Multiple file playback is supported. The documentation is a bit sparse and setup is not straight forward. However due to the simple nature of the program start up time can be 15-30 minutes. Cowlog would be an effective software for projects requiring very simple logging of events. However there are some severe limitations in the project set-up and execution.
· Use of VLC and Mplayer provide a wide array of supported video types.
· Simple interface make for easy start-up and use.
· Multiple file playback
· Large, movable, scalable playback screens allow for effective use of multiple monitors.
· Small coding interface can be moved and placed over the playback windows allowing for more effective use of the screen space.
· Only events codes allowed. State codes would have to be derived in the data analysis.
· Documentation was a bit sparse, leading to confusion around setting up a project. Setting up event code modifiers was not straight forward.
· Once a project was set-up, you cannot reconfigure the settings.
· No playback hotkeys
· Event hotkeys are not user programmable, you are stuck with alt-1, alt-2, … which are not easy.
· Multiple file playbacks are synchronized to the start time for each video, so differences in each videos start time (compared to clock time) will result in differences between videos.
· Events are logged (in seconds) from the start of the first video. No clock offsets.
· No ability to go back through a video session after it has been closed.
· Data is exported as a csv file logging the time of each event. Interpretation of the data would require post-processing of the raw event files.
File Formats Supported: Longomatch is powered by ONEPLAY, which supports most codecs.
LongoMatch has a really nice user interface and appears well laid out; however it suffers from the fact that it was designed for soccer. They have tried to generalize the software to work with other team sports as well as other applications, but the fundamental flow of the program stems from that original purpose. Longomatch can be used for non-intended purposes however data collection is limited to event code. Additional thought would have to be put into setting up the analysis to cope with a lack of general features. LongoMatch is well documented with a short, straight forward webpage to get started. Getting started with Longomatch took ~30 minutes.
· Simple to get started, good documentation
· Simple event codes with modifiers, easy to use with programmable quick keys.
· Good user interface, large event code buttons on the bottom of the screen, a timeline window allowing for viewing and modification of collected data.
· All the logged events, or a subset of a certain type of event can be exported as a highlight reel.
· LongoMatch has an interesting function which may be useful if modified appropriately. When coding event you can use a map of a soccer field to document the position and trajectory of the event. However this information was not exported in the datafile.
· No measurement options.
· Only single video playback.
· Only single point events can be logged. State events can be modified manually in the timeline window.
· When an event is logged in the video a secondary event window pops-up. This maybe helpful if you want to log secondary information (i.e. species, heading…) however if no secondary information is required then clicking the OK button via the mouse severally slows the work flow.
· The export features were a bit lacking. The excel export did not work. The csv export did not import correctly into excel. Renaming the file into a text file worked best.
File Formats Supported: AVI only
Ethowatcher is a very basic piece of software for coding and tracking animal behavior. I like that this software has two modules, one for ethnographic coding and one for tracking/measuring behavior. Having the ability to both code and measure behaviors is powerful, however these modules are separate and not integrated together. Overall, this system has more limitations then advantages when compared to similar systems. The ethnographic coding section is limited compared to other system listed above and not recommended. The tracking module on the other hand is novel and can be useful is certain controlled environments to automate data collection.
· Quick start-up with a built in tutorial (~30 minutes)
· The tracking module automatically measures animal positions, distance traveled, speed and area. However this requires a very controlled, calibrated environment.
· Good exporting to csv
· Only supports AVI’s with 320x240 resolution
· Single video playback
· No quick keys, no video playback functions, no playback speed adjustments
· Event codes only, no state codes. Furthermore every observation needs a subject. This is OK for studying individual animals behavior but annoying if you are just coding a video with no specific individuals of interest.
· No code modifiers
· No timing offsets. All codes are relative to the start (t=0) of the video clip.
Quantitative video measurement tools offer different functions than those for behavioral coding. These tools focus on measuring variables like point locations, direction/headings, distances, angles, durations and/or speed. While most of these behaviors can be categorically defined, the ability to measure these variables as continuous values can increase the resolution and statistical power of the analysis.
Before using these tools, there are several issues and limitations that should acknowledged with these type of analysis. Due to the 2-D projection of the image on the imaging sensor, the third dimension (Z) is lost or compressed. This makes all video data inherently two dimensional. All measurements are done in pixels. Converting pixel dimensions to real-world dimension is dependent on scale. This can make large objects far from the camera look the same size as small objects near the camera. Translating from pixel dimensions to real-world dimensions requires knowledge of the distance to the measurement plane. One assumption is that measurements are taken on a plane perpendicular to the camera. This limits the measurements that can be taken and can add measurement error when the measurement plane is not perpendicular to camera. If the distance from the camera to the measurement plane is not known then all measurements acquired from this plane are relative. If two objects are measured on planes at different distances from the camera then these relative measurements are not on the same scale and are not directly comparable. Lastly, if the distance to the measurement plane is known, or if an object of known size on the measurement plane is known, then the plane can be calibrated to a real-world coordinate system. These assumptions and limitations should be taken into account in the experimental design and in the interpretation of the data.
Three software packages were evaluated. All three had similar functions and workflow. Of these three, none of them were perfectly designed for animal behavior studies; however both Tracker and Kinovea are good pieces of software and are good tools for video analysis. I found Dartfish to be too specialized in sports analysis, plus the functionality was limited for the cost and compared to the other software packages.
File Formats Supported: MOV, AVI, FLV, MP4, WMV
Tracker was developed by Douglas Brown at Cabrillo College as a project funded through the Open Source Physics program. This software focuses on measuring, deriving and calculating the real-world physics of objects through video analysis. As such, it has some powerful tools to measure features like positions, headings, distances and angles. The software is a bit slow (especially with large data files) and not streamlined for large numbers of individual observations (like measuring tracks for many fish). However, due to the limited software options for quantifying these types of features, Tracker is definitely a good tool to keep on the hard drive.
· Supports the Xuggle and Quicktime media playback engines providing a wide range of import options.
· A series of video tutorials get the user up and working in under an hour.
· A user defines the axis planes and calibrates the coordinate system for relative or real-world measurements.
· Measures object positions, headings (through the use of vectors, or sequential points), distances and angles.
· Robust autotracker to automate position tracking of objects
· In program data processing, analysis and plotting.
· Adjustable step-rate for frame-by-frame analysis
· Kinematic or dynamic particles models derived from data.
· Curve-fitting of data.
· Data can be exported to a project/software specific *.trz file or raw data can be exported a text file.
· Single video playback
· No real-world clock offset, all measurements based on video start (t=0)
· Slow with big files
· The video reset every time I initiated a new track object
· Not well set-up to handle large amounts of individual objects to track
· Can only export one object track per text file
· Not great video playback options
File Formats Supported: AVI, MPG, MOV, WMV, MP4, FLV, 3GP, MKV, VOB, MOD, TOD
Kinovea would not be used for event/behavioral coding, instead Kinovea is a measurement tool which could be used to study direct behaviors, biomechanics or kinematics. Kinovea was designed to study the mechanics and ergonomics of athletes but these tools could be used in biological situations. Good documentation allows for start times to be between 30-45 minutes. Kinovea’s tool focus on tracking objects of interest and calculating the distance, speed and angles of such objects. Good export utilities make this a nice piece of software.
· Good documentation both online and within the help browser get the user up and running in 30-45 minutes.
· A wide range of support for various file formats
· Dual videos with manual synchronization between the two videos
· A zoom window to highlight areas of interest
· Ability automatically track objects and calculate a track path. The tracking function appears to track the objects of interest well between frames.
· Stopwatch function can be used to measure event duration.
· The tracking tool combined with the stopwatch tool integrates to calculate the distance traveled and speed of a track.
· Data is exported into an excel file. Post-processing will be required.
· When attempting to use two videos in dual view mode, the program frequently crashed. May or may not be a problem for other people.
· It would be slow if you have to measure many objects. Ok but not the quickest workflow.
· Would require a fixed, calibrated environment to gather real-world data on speed and distances.
File Formats Supported: Not apparent in the documentation.
Dartfish is a commercial product designed for sports analysis. Dartfish has many modules, offering a diversity of functions. The basic Dartfish package focuses mainly on annotating video files. Additional packages allow for additional functionality including automatic object tracking, event coding and advanced video procedures. However these functions come with a price. Dartfish is well documented however due to the extensive documentation start-up was relatively slow (1-2 hours). Overall, I found the functionality of Dartfish marginal. Most of the functions within Dartfish are also in Kinovea or Tracker, which is free. More advanced functions, like automated tracking have to be purchased as a separate module. Additionally, there is no easy way to export data for additional analysis.
· Well documented
· Multiple video playback (up to 4 videos simultaneously) with manual synchronization. Videos can be played in split screen mode or blended overlay mode.
· Zoom tool on each video
· Multiple videos can be stacked vertically or horizontally
· Stopwatch, distance and angle measurements.
· Manual object tracking (auto tracking available for purchase). Can track points, splines and angles.
· Written and oral annotation.
· Key positions (only on a single video at once)
· Additional toolboxes for tagging and advance video processing available for purchase.
· Slow start-up time given the functionality (1-2 hours).
· Extra functionality, extra cost.
· No ability to export measurements.
· Data collection would be slow.