This study investigates the equivalence or compatibility between U-Net and visual segmentations of fibroglandular tissue regions by mammography experts for calculating the breast density and mean glandular dose (MGD). A total of 703 mediolateral oblique-view mammograms were used for segmentation. Two region types were set as the ground truth (determined visually): (1) one type included only the region where fibroglandular tissue was identifiable (called the 'dense region'); (2) the other type included the region where the fibroglandular tissue may have existed in the past, provided that apparent adipose-only parts, such as the retromammary space, are excluded (the 'diffuse region'). U-Net was trained to segment the fibroglandular tissue region with an adaptive moment estimation optimiser, five-fold cross-validated with 400 training and 100 validation mammograms, and tested with 203 mammograms. The breast density and MGD were calculated using the van Engeland and Dance formulas, respectively, and compared between U-Net and the ground truth with the Dice similarity coefficient and Bland-Altman analysis. Dice similarity coefficients between U-Net and the ground truth were 0.895 and 0.939 for the dense and diffuse regions, respectively. In the Bland-Altman analysis, no proportional or fixed errors were discovered in either the dense or diffuse region for breast density, whereas a slight proportional error was discovered in both regions for the MGD (the slopes of the regression lines were -0.0299 and -0.0443 for the dense and diffuse regions, respectively). Consequently, the U-Net and ground truth were deemed equivalent (interchangeable) for breast density and compatible (interchangeable following four simple arithmetic operations) for MGD. U-Net-based segmentation of the fibroglandular tissue region was satisfactory for both regions, providing reliable segmentation for breast density and MGD calculations. U-Net will be useful in developing a reliable individualised screening-mammography programme, instead of relying on the visual judgement of mammography experts.

Echolocating bats perceive the surrounding environment by processing echoes of their ultrasound emissions. Echolocation enables bats to avoid colliding with external objects in complete darkness. In this study, we sought to develop a method for measuring the collective behavior of echolocating bats (Miniopterus fuliginosus) emerging from their roost cave using high-sensitivity stereo-camera recording. First, we developed an experimental system to reconstruct the three-dimensional (3D) flight trajectories of bats emerging from the roost for nightly foraging. Next, we developed a method to automatically track the 3D flight paths of individual bats so that quantitative estimation of the population in proportion to the behavioral classification could be conducted. Because the classification of behavior and the estimation of population size are ecologically important indices, the method established in this study will enable quantitative investigation of how individual bats efficiently leave the roost while avoiding colliding with each other during group movement and how the group behavior of bats changes according to weather and environmental conditions. Such high-precision detection and tracking will contribute to the elucidation of the algorithm of group behavior control in creatures that move in groups together in three dimensions, such as birds.

We propose a pose estimation method using a National Advisory Committee for Aeronautics (NACA) airfoil model for fish schools. This method allows one to understand the state in which fish are swimming based on their posture and dynamic variations. Moreover, their collective behavior can be understood based on their posture changes. Therefore, fish pose is a crucial indicator for collective behavior analysis. We use the NACA model to represent the fish posture; this enables more accurate tracking and movement prediction owing to the capability of the model in describing posture dynamics. To fit the model to video data, we first adopt the DeepLabCut toolbox to detect body parts (i.e., head, center, and tail fin) in an image sequence. Subsequently, we apply a particle filter to fit a set of parameters from the NACA model. The results from DeepLabCut, i.e., three points on a fish body, are used to adjust the components of the state vector. This enables more reliable estimation results to be obtained when the speed and direction of the fish change abruptly. Experimental results using both simulation data and real video data demonstrate that the proposed method provides good results, including when rapid changes occur in the swimming direction.

環境に設置されたセンサで獲得した情報から，公共空間や商業施設などを行き交う人々が構成するグループを検出できれば， そのグループに応じた情報提供が可能となり，さらには施設の利用状況を知る貴重な情報となる．グループの存在を知るにはお互いの距離や視線方向などが有用であるが，混雑していたり，グループが分かれて行動しているときなどでは十分ではない．我々はこのような場合に有用な特徴として，人物間のインタラクションを示すジェスチャーに着目した歩行者グループ検出手法を提案する．提案手法では，防犯カメラで捉えた画像のオプティカルフローの変化でジェスチャーを検知し，人物間距離や視線方向にこの検知結果を加味してグループを検出する．実環境で収集したデータを用いた評価実験を行い，ジェスチャー情報を利用するとグループの見落としが低減できることを確認した．

Gait is known as one of biometrics, and there have been many studies on gait authentication. In those studies, it is implicitly assumed that the gait of a certain person is always constant. It is, however, untrue in reality a person usually walks differently according to their mood and physical/mental conditions, which we call "inertial states."Motivated by this fact, we organized the research project "Behavior Understanding based on Intention-Gait Model", whichwas supported by JST-CREST from 2010 to 2017. The goal of this project was to map "gait", in the broad sense of the term, to inertial states such as attention, social factors, and cognitive ability. In this chapter, we provide an overview of the three kinds of estimation technologies considered in this project: attention, social factors, and cognitive ability.

If we can detect groups of visitors in public spaces and commercial facilities, we can provide information depending on the attributes of the groups, and we can also provide statistics with regards to the usage of the facilities for the owners of the facilities. The features, such as person-to-person distance and gaze direction, is useful for group detection and have been used in a number of works. However, almost all of the works extract the features from the whole data. This causes miss-detection in some cases. Even when we are walking with a fiend or colleague, we do not interact with the others all of the time. This means that a meaningful information for group detection is embedded within a part of timeseries data, not all of the data. We have to pick up the meaningful information and ignore the others. To this end, we divide whole of the time-series data into a set of data along the time axis. We apply the multiple instance learning (MIL) to find out the meaningful information among the data. The features computed for each time slot are treated as instances in MIL. MIL can extract one or more positive instance from all of the instances, including positive and negative ones. We conducted experiments using two types of data: simulated group actions and actual actions. Our method outperforms the existing method for both of the data.

As a method for supporting fish farming, this paper presents a video scene detection when farmed fry start instantaneously in a tank due to environmental stimuli. Although some environmental stimuli such as sound noises or lighting startle the fry and the stimuli bring about the instantaneous response, actual situations around the tanks in which the stimuli occur are unclear in detail. From the fact the fry often die due to crashes to the tank's wall and between the fry by the response, a monitoring system for the fry and situation around the pool could find causes of the stimuli, and it could result in decrease of the death number of the fry. In this research, the fry which swim in a tank are monitored by a video cam- era and the video scenes at the response are detected by a SVM with a feature value which represents fry's acceleration using sequential frames of the moving image. Preparing the moving images which include scenes of the response by fish in a tank, performances of the proposed method were examined. From experimental results, accuracy ratios of the recall and the precision for the scene detection have shown more than 80% on average and 100% under normal illuminances (108.5 lux on average), respectively.

We propose a visual tracking method with an NACA airfoil model for dense fish schools in which occlusions occur frequently. Although much progress has been made for tracking multiple objects, it remains a challenging task to track individuals due to factors such as occlusion and target appearance variation. In this paper, we first introduce a NACA airfoil model as a deformable appearance model of fish. For occluded fish, we estimate their positions, angles, and postures with template matching and simulated annealing algorithms to effectively optimize their parameters. To improve performance of tracking, we repeatedly track fish with the parameter estimation algorithm forwards and backwards. We prepared two real fish scenes in which the average number of fish is over 25 in each frame and multiple fish superimpose over 50 times. Experimental results for the scenes show that fish are practically tracked with our method compared to a tracking method based on a mixture particle filter. Over 75 % of fish in each scene have been tracked throughout the scene, and the average difference is less than 4 % of the mean body length of the school.

We have developed a screen sharing system to share contents between two persons, and applicable to one-to-one remote teaching. The screen of a PC at one side is shared with the screen of a tablet at the other side through the network to convey instructions from an operator to a collaborator. This system makes it possible to arbitrarily select a part of the screen of the PC at the operator side. The selected screen is presented in the tablet at the collaborator side. The collaborator can adjust the scale of contents, and capture the screen. By analyzing such operations, the system can understand and record which parts of the contents the collaborator paid attention to. In addition, the camera of the tablet can be used as a simple scanner to digitize paper documents easily. Further, characters and symbols drawn with a finger or a pen on the tablet screen can be presented on the PC at the other side.

When using multiple Kinects, there must be enough distances among neighboring Kinects to avoid spoiled range data caused by the interference of their infrared speckle patterns. In the arrangement, their overlapped regions are too small to apply existing calibration methods using correspondences between their observations straightforwardly. Therefore, we propose a method to calibrate Kinects without large overlapped regions. In our method, first, we add extra RGB cameras in an environment to compensate overlapped regions. Thanks to them, we can estimate their camera parameters by obtaining correspondences between color images. Next, for accurate calibration, which considers range data as well as color images of Kinects, we optimize the estimated parameters by minimizing both the errors of correspondences between color images and those of range data of planar regions, which exist in a general environment such as walls and floors. Although our method consists of conventional techniques, its combination is optimized to achieve the calibration. (C) 2015 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

We propose a visual tracking method for dense fish schools in which occlusions occur frequently. Although much progress has been made for tracking multiple objects in video images, it is challenging to track individuals in highly dense groups. For occluded fishes, estimation of their positions and directions is difficult. However, if we know the number of fishes in a local area, we can accurately estimate their states by matching all of the combinations of possible parameters on the basis of our appearance model. We apply the idea to track multiple fishes in a school. Experimental results show that multiple fishes are practically tracked with our method compared to a well-known tracking method, and the average difference is less than 4% of the mean body length of the school.

When we act in a group with family members, friends, colleagues, each group member often play the respective role to achieve a goal that all group members have in common. This paper focuses on leadership among various kinds of roles observed in a social group and proposes a method to estimate a leader based on an interaction analysis. In order to estimate a leader in a group, we extract pointing actions of each person and measure how other people change their actions triggered by the pointing actions, i.e. how much influence the pointing actions have. When we can see the tendency that one specific person makes pointing actions and the actions have a high influence on another member, it is very likely that the person is a leader in a group. The proposed method is based on this intuition and measures the influence of pointing actions using their motion trajectories. We demonstrate that the proposed method has a potential for estimating the leadership through a comparison between the computed influence measures and subjective evaluations using some actual videos taken in a science museum.

This paper proposes a method of camera calibration that compares the appearance of two images. Unlike conventional methods that evaluate point-to-point correspondences, ours makes a dense evaluation of the correspondence between two images. This enables us to robustly and efficiently calibrate range finders that are camera based. We explain the main principles and algorithm underlying our method, and we also present the results obtained from simulations and experimentally obtained data.

This paper proposes a novel method to reconstruct dynamic scenes by integrating depth data obtained by multiple Kinects, which cannot be synchronized to one another In this method, the multiple Kinects located so as to cover the whole surface are firstly calibrated so that their depth data are mapped into the world coordinate system. The synchronous depth data for each Kinect is then generated by interpolation of temporally neighboring captured data. Experimental results of marching person reconstruction show the effectiveness of our method.

In this paper we propose a method for temporal interpolation of a point cloud undergoing occlusions and topological changes. The point cloud is first merged into fine clusters, which are then further merged into coarse clusters for each source and target shape. In conjunction with trash box bins to cope with occlusions, a coarse correspondence between a source and a target shape is found that minimizes the transportation cost in the earth mover's distance framework. Subsequently, a fine correspondence is found in a similar way based on the coarse correspondence constraint to suppress locally isolated motion. Finally, the source and target point clouds are transported based on the fine correspondence. Experiments with point cloud sequences captured by a Kinect range finder show promising results.

This paper proposes a depth measurement error model of consumer depth cameras such as Microsoft KINECT, and its calibration method. These devices are originally designed for video game interface, thus, the obtained depth map are not enough accurate for 3D measurement. To decrease these depth errors, several models have been proposed, however, these models consider only camera-related parameters. Since the depth sensors are based on projector-camera systems, we should consider projector-related parameters. Therefore, we propose the error model of the consumer depth cameras especially the KINECT, considering both intrinsic parameters of the camera and the projector To calibrate the error model, we also propose the parameter estimation method by only showing a planar board to the depth sensors. Our error model and its calibration are necessary step for using the KINECT as a 3D measuring device. Experimental results show the validity and effectiveness of the error model and its calibration.

Existing background subtraction methods often fail to extract a foreground region whose color is similar to that of the background. When we use a co-located camera and range sensor, by which we can obtain both a color image and depth map simultaneously, it is expected to get a better foreground region by integrating the two kind of images. However, it is not straightforward when a moving object is observed because the camera and range sensor do not capture the scene synchronously. In this paper, we propose a novel method that pseudo-synchronize the camera and range sensor and integrate the background subtraction of the color and depth images to realize a good foreground extraction. Experimental results of a walking human show its effectiveness.

In this paper, we propose a method for producing an informative wide-range composite image of a target object from a video taken by a camera moving parallel to the ground plane. Generally, when one generates an image that presents the scene in a video, it is difficult to select the information to be shown in the generated image from the large amount of information contained in the video. Our method involves the use of trajectories of feature points for selecting informative regions and generates a composite image on the basis of the trajectories. First, we create a 3D volume by piling up frame images on a time line. Second, we set a curved surface in that volume. Finally, a wide-range image is formed by considering the pixels along the curved surface. Our method selects a curved surface that passes through a large number of feature points of the target object. Therefore, we obtain an informative wide-range composite image of the target object from a video. By performing experiments with some actual videos, we verified that our method can generate an informative wide-range image of the target object effectively.

We describe a method for generating an informative wide-view image using images captured by a moving camera. The generated image allows for events in the scene observed by the camera to be understood easily. Our method does not use 3D shape information explicitly. Instead, it employs the trajectory of feature points across multiple images and generates a composite image by taking into account the distribution of the trajectories of the feature points. © 2010 IEEE.

This paper proposes a method for acquiring the prior probability of human existence by using past human trajectories and the color of an image. The priors play an important role in human detection as well as in scene understanding. The proposed method is based on the assumption that a person can exist again in an area where he/she existed in the past. In order to acquire the priors efficiently, a high prior probability is assigned to an area having the same color as past human trajectories. We use a particle filter for representing and updating the prior probability. Therefore, we can represent a complex prior probability using only a few parameters. Through experiments, we confirmed that our proposed method can acquire the prior probability efficiently and use it to realize highly accurate human detection. © 2010 Information Processing Society of Japan.

Large-print textbooks are used by low vision students in school. Because these books are mainly prepared by volunteers and almost all steps in the preparation process are performed manually, they cannot be mass-produced. The chronic shortage of these books has been a social problem in Japan. The procedure for preparing a large-print textbook involves (1) converting the size of figures and characters in the original textbook to one suitable for low vision students and (2) positioning them appropriately on the pages. In this paper, we propose a novel method for automatically optimizing a layout by employing a context structure. We represent the context structure by using a graph called a context structure graph. The proposed method first allocates each material to an appropriate page, and then optimizes the layout of each page by using the sequence-pair method. Throughout these operations, we employ an objective function derived from the context structure graph to ensure that the context in the original textbook is preserved in the large-print textbook prepared. © 2009 IEEE.

This paper reports a method for acquiring the prior probability of human existence by using past human trajectories and the color of an image. The priors play important roles in human detection as well as in scene understanding. The proposed method is based on the assumption that a person can exist again in an area where he/she existed in the past. In order to acquire the priors efficiently, a high prior probability is assigned to an area having the same color as past human trajectories. We use a particle Iter for representing the prior probability. Therefore, we can represent a complex prior probability using only a few parameters. Through experiments, we confirmed that our proposed method can acquire the prior probability efficiently and it can realize highly accurate human detection using the obtained prior probability.

The increasing number of cataract sufferers is a serious problem because cataracts are a leading cause of blindness in the world. To avoid blindness from cataracts, we need to detect them early. Today, ophthalmologists use a slit lamp to diagnose cataracts. This equipment is expensive and requires special training to use it. Unfortunately, a lot of developing countries have a limited number of ophthalmologists and health facilities, while a lot of cataract sufferers live in developing countries. Our algorithm is aimed at such developing countries and rural areas, and its main equipment is a compact digital camera. This equipment is inexpensive and easy for anyone to use without special training. Examination can be easily carried out in an outpatient department. We have already proposed a cataract screening method for the equipment that considers about specular reflection and statistical texture appearance inside the pupil. However, in order to accomplish our goal, not only the screening method itself, but also other aspects such as compactness and ease of use are crucial. In this paper, we will discuss system design for a compact cataract screening system. The final goal of our research is developing a simple and robust screening system for cataract with a compact digital camera. A user simply takes a patient's photograph and the system will automatically analyze the image by extracting information inside a pupil, including specular reflection analysis and texture appearance analysis, for distinguishing between serious and non-serious conditions. We emphasize how to implement our system in real conditions by giving a recommendation for acquiring appropriate image data while taking a photograph so that our system will work well. The results in a real implementation are similar to our recommendation.

We propose a method that analyzes interaction between pedestrians based on their trajectories obtained using sensors such as cameras. Our objective is to understand the mutual relationship between pedestrians and to detect anomalous events in a video sequence. Under such situations, we can observe the interaction between a pair of pedestrians. This paper proposes a set of features that measures the interaction between pedestrians. We assume that a person is likely to change his/her walking patterns when he/she has been influenced by another person. Based on this assumption, the proposed method first extracts the transition points of a walking pattern from trajectories of two pedestrians and then measures the strength of the influence using the temporal and spatial closeness between them. Finally, experimental results obtained from actual videos demonstrate the method's effectiveness in understating mutual relationships and detecting anomalous events.

没入型投影ディスプレイシステムなどで用いられる凹面スクリーン上に映像を投影したとき,相互反射や表面反射特性の影響で,入力された映像の明暗パターンがひずんで鑑賞者へ提示される.本論文では,これらの影響を補償し意図する映像を忠実に再現する手法を提案する.原理的には,プロジェクタ(光源)からスクリーン,鑑賞者とたどるライトトランスポートを求め,その逆変換を施すことで反射ひずみ補償が達成され,いくつかの例が提案されている.しかし,それらは本来面積をもった画素同士の対応であるライトトランスポートを,点同士の対応関係として近似的に定式化しているため,近似誤差によって反射ひずみを正しく補償できない場合がある.提案手法は,単純な点対応で生じる,単位入射光を投影したときの直接反射と相互反射の境界部分の急激な明度変化が観測できないという問題に対し,プロジェクタとカメラの間の幾何的対応関係をとり,幾何変換を行った上でライトトランスポートを求める.これにより,凹面スクリーンの幾何形状に応じたコンパクトなライトトランスポートで反射ひずみ補償が可能となった.

The high prevalence of cataracts is still a serious public health problem as a leading cause of blindness, especially in developing countries with limited health facilities. In this paper we propose a new screening method for cataract diagnosis by easy-to-use and low cost imaging equipment such as commercially available digital cameras. The difficulties in using this sort of digital camera equipment are seen in the observed images, the quality of which is not sufficiently controlled; there is no control of illumination, for example. A sign of cataracts is a whitish color in the pupil which usually is black, but it is difficult to automatically analyze color information under uncontrolled illumination conditions. To cope with this problem, we analyze specular reflection in the pupil region. When an illumination light hits the pupil, it makes a specular reflection on the frontal surface of the lens of the pupil area. Also the light goes through the rear side of the lens and might be reflected again. Specular reflection always appears brighter than the surrounding area and is also independent of the illumination condition, so this characteristic enables us to screen out serious cataract robustly by analyzing reflections observed in the eye image. In this paper, we demonstrate the validity of our method through theoretical discussion and experimental results. By following the simple guidelines shown in this paper, anyone would be able to screen for cataracts.

画像計測において,単独の画像では得られない情報を補う手法として,多重画像の利用が広く行われている.その中で,本論文では多重光源画像を用いた2種類の画像計測手法を提案する.多重光源画像を用いた従来手法では,正確な形状計測を行うため,表面の反射特性の厳密なモデル化や大掛りな装置が必要とされており,実環境への適用は容易ではない.それに対し本論文では,精密なモデルを用いず,多重光源画像での明度変化に着目して計測対象の大まかな三次元情報を計測する手法を提案する.まず第一の手法では,光源位置を変化させたときの画素値の変化からトンネル壁画のひび割れを検出する.第二の手法では,鉄道道床でのステレオマッチングの精度を向上させるために,様々な方向から照明を照射した画像を統合して利用する.本論文ではこれらの手法の原理及びアルゴリズムを述べ,実画像を用いた実験でその有効性を示す.

Background subtraction is widely used as an effective method for detecting moving objects in a video image. However, background subtraction requires a prerequisite in that image variation cannot be observed, and the range of application is limited. Proposed in this research paper is a method for detecting moving objects by using background subtraction that can be applied to cases in which the image has varied due to varying illumination. This method is based on two object detection methods that are based on different lines of thinking. One method compares the background image and the observed image using invariant features of illumination. The other method estimates the illumination conditions of the observed image and normalizes the brightness before carrying out background subtraction. These two methods are complementary, and highly precise detection results can be obtained by ultimately integrating the detection results of both methods. © 2006 Wiley Periodicals, Inc.

3-D video records dynamic 3-D visual events as is. Applications of 3-D video include wide varieties of human activities. For promoting these applications in everyday life, a standardized compression scheme for 3-D video is required. We developed a practical and effective scheme for representing and compressing 3-D video named "skin-off," in which both the geometric and visual information are efficiently represented by cutting a 3-D mesh and mapping it onto a 2-D array. Skin-off has much in common with geometry images proposed by Gu et al. However, while geometry images use only 3-D surface shape information to generate 2-D images, skin-off uses both 3-D shape and texture information to generate them. This enables us to achieve higher image quality with limited bandwidth. Experimental results demonstrate the effectiveness of skin-off.

背景差分は, 画像中の移動対象を検出する有効な手法として広く利用されている.しかし, 背景差分を行うには, 背景部分での画像の変化が観測されないという前提条件が必要であるため, その適用範囲は限られている.本論文では, 照明変化による背景部分の画像の変化が起きた場合にも適用可能な背景差分による移動対象検出法を提案する.本手法は, 異なった考え方に基づく二つの対象検出法に基づいている.一方は, 照明に不変な特徴を用いて背景画像と観測画像の比較を行う手法である.他方は, 背景差分を行う前に観測画像の照明条件を推定し, 輝度の正規化を行う手法である.これら二つの手法は互いに相補的であり, 最終的に双方の検出結果を統合することで精度の高い検出結果を得ることができる.実験の結果, 実画像に対する本手法の有効性が示された.

鳥や魚などの生物の集団として振る舞いは長く研究対象として注目されてきたが，群行動を精緻に計測し，群れのナビゲーションのメカニズムを解明する試みは限られている．そこで，群になって行動している生物を撮影した映像からその動きを獲得するための「汎用映像処理プラットフォーム」を構築してきた．本年度の研究実績の概要は以下のとおりである． (1) 魚の検出・追跡に関する基礎技術の構築 まず，水槽の中を遊泳する魚の検出を行うために，環境変化に対応して魚を高精度で検出する手法を考案した．この手法では水槽に設置されている器具や水面の照り返しで水面が観測できない領域を泳ぐ魚の様子を周囲から予測する機能も実装している．また，魚の動きをより詳細に知るために，その姿勢をパラメトリックに表現するモデルを導入し，そのパラメータを精度良く推定する方法を考案した．この手法は姿勢の時系列変化をモデル化できるので，異常検知や姿勢変化の予測への活用が期待できる． (2) コウモリの生態観測のための3次元計測 新学術領域「生物移動情報学」に参加している生態学を専門とする研究者と協力して，飛翔するコウモリの3次元軌跡を推定する手法を考案した．この手法では洞窟から出現するコウモリをステレオカメラで計測し，画像処理による検出，カメラ間の対応付け，3次元軌跡の復元を行っている．これによって，トンネルから出現するコウモリの尾数の傾向や，複数のコウモリが協調して飛行する様子を詳細に観測できるようになると期待される．また，コウモリが飛翔している暗い空間の中でもその位置を精度良く得るための手法を考案し，その基本的有効性を確認した．

In this study, we developed and verified an image processing technique focusing on the change in facial orientation and mainly applying it to the detection of suspicious persons. For face detection and face change estimation, we used TinyFace and Deepgaze, both of which are considered to have the highest level of accuracy in their respective fields, to estimate the intentions of the person, especially the person acting suspiciously, using the time series of face orientation data obtained from TinyFace and Deepgaze. From the time series data, we chose to identify people who moved strangely or unnaturally large in comparison with other people, and identified them as people who were acting suspiciously. In order to verify the effectiveness of this method, we conducted experiments with a video dataset assuming spectator seats in a stadium.

This project is aimed at developing a new image-based 3D shape and motion capture technique for underwater environment using multi-view projector-camera system, and its achievements include a new underwater camera model for modeling refractive process and its application for an efficient numerical forward projection, a new catadioptric system for micro-scale object capture, and underwater object detection and tracking algorithms. In particular the project developed a new teleidoscopic imaging system which integrates a spherical lens and a kaleidoscope in front of a perspective camera to realize a multi-view capture of micro-scale objects from a single viewpoint.

課題A:大規模群泳行動の自動計測 本研究では，(a) 映像中で群になって泳ぐ魚を検出・追跡する技術，(b) 検出した魚の姿勢をモデル化し時間経過に伴うその動きを獲得する技術，の開発を行った．2018年度はマイワシはクロマグロだけでなく，コウモリやメダカ，人物などへの適用を行った．現状では(b)の技術の適用先はマイワシに限定されているが，(a)はクロマグロ成魚などの大きな個体に対しても対応できるように拡張し，様々な対象に対して有効に動作することを確認した． 課題B:群泳行動データセットの構築 水族館や水産養殖場で収集した映像を用い，(a)映像データ，(b)それぞれに映っている個体の位置，を対にしたデータセットの構築を行った．対象は先述のクロマグロ成魚・クロマグロ稚魚・マイワシとし，それらについて位置データを人の眼で付与し，それを組みとしてデータセットとした．2018年度はこれまでに不足していた環境条件（暗い場合やバースト現象のバリエーション）を補うことを念頭におき，データセットを拡張した． AとBに共通して「生物移動情報学」の他の研究班との協力を密にして実施した．先のコウモリに映像解析はその一例である． また，これらの成果を広く一般にアピールするために，佐世保市の九十九島水族館 海きららにおいて，2018年9月に一般向けの研究発表会「九十九島の日 研究発表会」を九十九島水族館と共同で実施した．40名の参加者を得て好評のうちに終了した．

Image- and video-based object recognition usually requires a large amount of training data for sufficient performance. However, it is not always possible to obtain such a large dataset. Hence, we propose a method for extracting useful classifiers from a set of existing classifiers that is obtained by using training data of other scenes. Combining the extracted classifiers enables us to achieve better performance than solely using training data of a new scene. Experimental results show the effectiveness of the proposed method.

We propose an object detection framework especially for wide-area visual surveillance system. Existing methods usually make use of features which target objects usually have in common. However, those common features are sometimes insufficient for robust detection. Therefore, in this project, we model contextual information which isparticularly associated with a certain location. This enables us to detect targetsrobustly and efficiently.

In order to detect and track pedestrians in an outdoor scene, we proposed the methods that exploit its geometric structure. For example, if we know where a road and a building are in the scene, it is possible to improve the performance of pedestrian detection drastically. Specifically speaking, we proposed the methods for (1) estimating prior probability of human existence and (2) obtaining accurate human motion trajectories. Additionally, in order develop the new application area of pedestrian detection and tracking, we proposed a method that analyzes interaction between pedestrians and es...

Development of stable and dependable streaming method: To satisfy high-availability and low-cost at the same time, it is effective to connect multiple different cheap and best-effort networks and realize a feature to select a network among them or use them at the same time. UDP which is used for streaming does not have mechanisms to control bandwidth or recognize situation of network. We use TFRC (TCP Friendly Rate Control), which has mechanism for congestion control, for streaming using multiple networks at the same time. We also use SCTP (Stream Control Transmission Protocol) not to inter...

In the 21st century, our personal and social activities are conducted in two different domains: physical real world and cyber network society. For realizing smooth and casual integration of these domains, we need a novel interaction model that goes a step further beyond the existing "command-and-response model."In this research, we proposed a concept of "man-machine symbiotic systems" as the next generation interaction model. Man-machine symbiotic systems are characterized by such a feature that the systems work for humans even if they are not explicitly ordered. For rea...

昨年度に行った検討では,画像パターンの微小な変化が線形モデルに従うときに,背景と対象と識別が可能であることを示したが,本年度は,任意の画像パターンに対してその変化を効率的に記述する手法の検討を行った.この手法では,画像パターンの自己相関に着目し,画像上で観測された変化と自己相関を比較することで,観測された変化が動的シーンでみられる揺らぎによるものなのか,検出対象によるものなのかを識別する.従来の手法では,処理対象の画像が持つ2次元空間構造,すなわち2次元の隣接関係には着目せずに,画像を単なる1次元データとして扱うものが多く,そのために性能に限界があったと考えられる.これに対し,本手法で着目する自己相関は2次元空間構造を現したものであり,これを利用することで精度の向上が期待される.本年度はこの考え方に基づく手法の基本的な有効性を確認した.この考え方は,動的環境下の対象検出のみならず,画像解析の様々な面に応用が可能である.その一例として,画像同士のマッチングにおけるサブピクセル単位での変位推定に応用を試みた.対象検出が,画像に発生した変動が対象によるものかどうかを識別する問題であったのに対し,サブピクセル推定は画像の変動からその変位を精密に推定する問題と位置づけることができる.すなわち,平行移動による画像の変化を精密にモデル化する,という面では本質的に同様の問題であるといえる.こ...

We have obtained the following technical attainments to realize 3D video generation, visualization, editing, and coding.1. PC cluster system (PC : 30, active camera : 25) to capture synchronized multi-view video of human action2. Precise calibration method for quasi fixed-viewpoint pan-tilt-zoom camera3. 3 base plane visual cone intersection algorithm and its parallel pipeline processing method for real-time dynamic 3D shape reconstruction using PC cluster ; with this system, we can reconstruct over 10 volumes per second of human action at 2cm X 2cm X 2cm spatial resolution.4. cooperative c...