VIDEO STABILIZATION BASED ON 
OPTICAL FLOW AND REGRESSION 
ANALYSIS  
MSC IN COMPUTER SCIENCE 
 
UNIVERSITY OF MORATUWA 
SRI LANKA 
 
DECEMBER, 2007 
S. ASHADA KARUNARATHNA 
 
 ii
 
  
 
Video Stabilization based on Optical 
Flow and Regression Analysis 
This dissertation was submitted to the 
Department of Computer Science and Engineering 
Of the 
University of Moratuwa 
in partial fulfillment of the requirements of the 
Degree of MSc in Computer Science specializing in 
Software Architecture 
Department of Computer Science and Engineering 
University of Moratuwa 
Sri Lanka 
 
December, 2007 
 
S. Ashada Karunarathna 
 iii 
 
 
 
Video Stabilization based on Optical 
Flow and Regression Analysis 
I declare that the work included in the dissertation 
in part or whole has not been submitted for any 
other academic qualification at any institution. 
 
 
 
 
 
 
 ………………………………… 
S. A Karunarathna( Candidate ) 
 
 
I certify the above statement of the candidate. 
 
 
 
 
………………………………….. 
Dr. Ajith Pasqual (Supervisor) 
 
 
 
  i
Abstract 
 
 
Video stabilization is an important video enhancement technology which aims at 
removing annoying unwanted motion from videos which is being used by various 
vision applications as a pre-processing activity. Since jitter levels cannot always be 
fully determined by systems automatically without affecting to the desired camera 
movements, it is essential to provide an easy way for ordinary users to decide what to 
be considered as jitter. 
 
A major problem of current general purpose software video stabilizers is removing 
some of the desired motions or leave undesirable motions in video streams because of 
the incapability in determining what to be considered as wanted/unwanted.  This 
dissertation describes a  practical software based solution for video stabilization that 
address this by providing a better control over to the user in determining level of 
stabilization willing to have in the output video.   
 
The proposed system initially determines to level of jitter automatically and provides 
high flexibility to the user to change the level of the jitter to be eliminated using few 
user parameters before generating full frame based stabilized output. This process is 
backed up by set of graph outputs that can be easily understood by ordinary user. 
 
Video is perceived as stable when the scene shown in consecutive frames has smooth 
flow over time. Based on this observation, proposed method removes horizontal and 
vertical undesirable movements while providing above flexibility using non-linear 
regression analysis based algorithm in iterative manner. 
 
 
 
 
  
  ii
Acknowledgements 
 
 
Having completed the MSc. Degree program at the Department of Computer Science 
and Engineering, University of Moratuwa, I would like to take this opportunity to 
thank those who helped me a lot in completing it at this level. 
 
First of all, I would like to thank my supervisors, Dr. Jagath Samarabandu, University 
of Western Ontario, Canada and Dr. Ajith Pasqual, University of Moratuwa, Sri 
Lanka for the supervision and guidance despite having busy schedules. Ms. Vishaka 
Nanayakara, as Head of the Department of Computer Science and Engineering, made 
sure that I had sufficient time and resources for research work although the 
department was constrained by the lack of staff. Also I would like to thank Dr. Sanath 
Jayasena for conducting the subject Research Seminar and the knowledge helped me a 
lot when browsing through the published research papers. 
 
My sincere thanks go to Codegen International ( Pvt ) Ltd for providing me the time 
for the research activities when ever needed. 
 
I should be grateful to all the staff members of the Department of Computer Science 
and Engineering for giving valuable feed backs during the progress review sessions. 
 
Finally I would like to thank my family for supporting me in every aspect in 
completing my degree program and the research successfully. 
  1
Table of Contents 
 
1 Introduction ...........................................................................................5 
2 Literature Survey ..................................................................................8 
2.1 Introduction ............................................................................................................ 8 
2.2 Existing Video Stabilization systems ..................................................................... 8 
3 Stabilization Scheme ........................................................................... 14 
3.1 Overview .............................................................................................................. 14 
3.2 Step 1 - Measure the transformations between adjacent frames .......................... 15 
3.3 Step 2 - Estimation of desired camera movements .............................................. 18 
3.4 Step3 - Compensation of each frame for unwanted motion ................................. 21 
3.5 Step4 - Reconstruction of undefined regions ....................................................... 22 
3.5.1 Achieving higher accuracy ........................................................................... 25 
4 Software base Implementation ........................................................... 28 
4.1 Functional requirements ....................................................................................... 28 
4.1.1 Use case diagram .......................................................................................... 28 
4.2 Detailed Design .................................................................................................... 29 
4.2.1 Class diagram ................................................................................................ 30 
4.2.2 Activity diagram ........................................................................................... 30 
4.3 Choosing Software libraries ................................................................................. 31 
4.4 User interface ....................................................................................................... 31 
4.4.1 Processing Video .......................................................................................... 32 
4.4.2 Generate Output Video ................................................................................. 34 
5 Experimental results and evaluation .................................................. 37 
5.1 Overview .............................................................................................................. 37 
5.2 Testing Strategy ................................................................................................... 37 
  2
5.3 Test input .............................................................................................................. 37 
5.4 Overview of results .............................................................................................. 38 
5.4.1 Statistical Analysis ........................................................................................ 38 
5.4.2 Perception of people ..................................................................................... 40 
5.4.3 Performance Analysis ................................................................................... 40 
6 Conclusion and Future work .............................................................. 41 
References ................................................................................................... 43 
Appendix (A) – Lucas Kanade Optical Flow Estimation ......................... 46 
Appendix (B) – Contents of the CD-ROM ................................................ 50 
  
  
  3
 
List of Figures 
 
Figure 1 Regression analysis to smooth out the camera movement. .................................. 21 
Figure 2 Adjusting frame to remove unwanted movement. ............................................... 22 
Figure 3 Calculating Stripe position ................................................................................... 26 
Figure 4 Use case diagram .................................................................................................. 28 
Figure 5 Class diagram ....................................................................................................... 30 
Figure 6 Activity diagram ................................................................................................... 30 
Figure 7 User Interface - Main Panel (before processing).................................................. 32 
Figure 8 User Interface – Process Video Panel .................................................................. 32 
Figure 9 Processing Mode - Frame with All Feature Points............................................... 33 
Figure 10 Main Panel – After processing the video ........................................................... 34 
Figure 11 Regression Analysis Graph – with Larger Noise Filter ..................................... 35 
Figure 12 Regression Analysis Graph – Smaller Chunk Size ............................................ 36 
Figure 13 Test results – Vertical component (For Input Video, Max filter size) ............... 38 
Figure 14 Test results – Vertical component (For Output Video, Max filter size)............. 39 
Figure 15 Image Pyramid ................................................................................................... 48 
 
  
  4
 
List of Tables 
 
Table 1 Algorithm overview ............................................................................................... 14 
Table 2 Algorithm - Measure the transformations between adjacent frames ..................... 17 
Table 3 Algorithm - Estimate the desired camera movements ........................................... 19 
Table 4 Algorithm – Iterative Regression Analysis............................................................ 21 
Table 5 Algorithm - Reconstruction of undefined regions ................................................. 24 
Table 7 Regression Analysis libraries evaluated ................................................................ 31 
Table 8 Processing Mode details ........................................................................................ 33 
Table 9 Test Input Videos ................................................................................................... 38 
Table 10 Configuration of Test computers ......................................................................... 40 
Table 11 Performance table ................................................................................................ 40