The analysis of deformable objects which have a high-degree of freedom has long been encouraged by numerous researchers because it can be applied to such diverse areas as medical engineering, video surveillance and monitoring, Human Computer Interaction, browsing of video databases, interactive gaming and other growing applications. Within the computerized environments, the systems are largely separated into marker based motion capture and markerless motion capture. In particular, markerless motion capture and analysis have also been heavily studied by numerous researchers using local features, color, shape, texture, and depth map from stereo vision, but it is still a challenging issue in the area of computer vision and computer graphics due to partial occlusion, clutter, dependency of camera viewpoints, high-dimensional state space and pose ambiguity within the target object. In this thesis, we address the issue of the efficient markerless motion capture and representation methodology using skeletal features for the purpose of analysis and recognition of their motion patterns in video sequences. To localize the motion of the target object in a 2D image and 3D volume, we extract the skeletal features by analyzing its Normalized Gradient Vector Flow in the space of diffusion tensor fields since skeletal features are more robust and efficient than other features in recognizing and analyzing the deformable object. The skeletal features within the target object are automatically merged and split by measuring the dissimilarity of tensorial characteristics between neighbor pixels and voxels. The split skeletal features are used as features in human action recognition to understand human motion and target object detection and retrieval for Content based Image Retrieval. This thesis provides the following contributions to the fields of computer vision and computer graphics: (i) it introduces the notion of the features in the space of diffusion tensor fields and evaluates the successful analysis method of such features for motion interpretation, (ii) it presents a theory and an evaluation of the methods for automatic skeleton splitting and merging with respect to similarity measure between neighbor pixels in two dimension or voxels in three dimension and, (iii) it presents and demonstrates our proposed principle methodologies for diverse applications such as human action recognition or sketch-based image retrieval. With our system we can robustly handle several computer vision tasks to recognize and understand the motion of the target object without any prior information. In particular, the human action recognition using 3D reconstruction from multiple images and the skeleton splitting procedure is firstly proposed in this thesis and shown to be a useful and stable methodology. Furthermore, users can easily express their intention by sketching the characteristics of a target object and derive available related objects from a data base by using our proposed method.