There has been much research on estimating noise and speech source direction, but there have not been many studies on estimating the source direction of instrumental sound sources. In this study, we considered the source direction estimation of a single instrumental sound. Direction estimation of sound sources by the multiple signal classification (MUSIC) method often causes large estimation errors. Then, we propose a technique for estimating the direction of musical instrument sound sources by applying regression analysis using a convolutional neural network (CNN), a type of neural network. We calculated the MUSIC spectrum obtained using MUSIC that uses the fundamental and harmonic components, which have relatively large amplitudes, and we estimated the direction of the sound source using the CNN with these components as input. We achieved this by focusing on the overtone structure of the instrumental sound source. This study demonstrated the effectiveness of this method using simulations in a monaural environment.

We have proposed an audio watermarking method based on modification of sound pressure level between channels. This method is focused on the invariability of sound localization against sound processing like MP3 and the imperceptibility about slightly change of sound localization. In this paper, we investigate about tolerance evaluation against various attacks in reference to IHC criteria.

This paper proposes a new regularization term of non-negative matrix factorization (NMF) for estimating pitch and onset time of electric bass. Both of them are indispensable information for automatic music transcription. However, it is difficult to estimate them from the electric bass performance because of trade-off problem between time resolution and frequency resolution. Applying the NMF simply to the electric bass performance, the correct note and its semitone are sometimes detected simultaneously. The proposed regularization term avoiding such miss detection based on the performance characteristic of the electric bass, that is, two notes are not generally played simultaneously. We define the regularization term in order to keep orthogonality of the adjacent activation decomposed by the NMF. As a result of the experiments using MIDI synthesizer-generated sound of electric bass and real played electric bass sound, it is confirmed that the F-measure of the proposed method is higher than that of comparative method.

We propose a watermarking method using nonnegative matrix factorization (NMF) for audio signals. NMF is applied to the host signal, and the amplitude spectrogram of the host signal is factorized into the basis matrix and the activation matrix, which are nonnegative matrices. The notes are estimated from the activation matrix, and the estimated notes are regarded as root notes. In the existing method, one-bit watermark is embedded into the dominant note corresponding to the root note. However, it is difficult to say that the embedding capacity is enough. Hence, in the proposed method, watermarks are embedded into the mediant and leading notes in addition to the dominant note. Up to three-bit watermarks can be embedded into one note while maintaining a detection accuracy.

In this paper, we propose an audio watermarking method using non-negative matrix factorization (NMF). The amplitude spectrogram of the host signal is decomposed the basis matrix and the activation matrix by NMF. These matrices are non-negative matrices and we use the activation matrix for embedding watermarks. Onset time and offset time are estimated from the coefficients of activation matrix and this interval is defined as note duration. The estimated notes are regarded as root notes and watermark are embedded by operating the activation coefficients of mediant or dominant note corresponding to root note.

A digital watermark is a technique to embed another digital data into digital contents such as music, image and movie. For audio signals, it is required that the sound quality of stego signal is not deteriorated. In existing methods, the high sound quality means that the difference between host signal and stego signal is small. On the other hand, in this paper, there is no problem even if a different sound from host signal is perceived as long as stego signal is maintained as music. Accordingly, we make a countermelody based on counterpoint and we propose a watermarking method that the countermelody is added to host signal as watermark signal. In this paper, we introduce not only chord tone but also nonharmonic note as an effective musical accent to watermark signal. Moreover, multiple notes can be arranged in one note of host signal, and it is expected to increase the embedding capacity of watermarks.

For audio signals, the sound quality of the stego signal should not deteriorate as much as host signal. With current methods, high sound quality means that the difference between the host and stego signals is small. We proposed a method with which high sound quality means that the sound quality of the stego signal is maintained as music even if another sound, except for host signal, is perceived. Accordingly, we focused on the counterpoint of music theory. A watermark signal is added in consonance to the tones of the host signal based on counterpoint. For this study, we used a diatonic chord as consonance. We also used a sound with an instrumental timbre as the watermark signal. The sound quality of the stego signal does not produce an unpleasant feeling to the listener even if the watermark signal is perceived.

In this paper, we described about a design of learning environment based on information structure and a realization of problem posing for learning disability. We design and developed software by modeling information structure of subject that is operated on software. In this research, we aimed at the domain of education and developed a learning environment for posing arithmetic word problem. Problem posing is well-known as effective learning. But students who have learning disability are impossible to learn by this exercise because they cannot pose the problem from scratch and cannot read complex sentence. However, learner is able to pose a word problem by selecting three cards in our software. So we assumed that these learners are able to learn by problem posing by using our learning environment. Experimental use is also reported.

Yajima et al. proposed a method based on amplitude and phase coding of audio signals. This method has relatively high sound quality because human auditory property is considered for embedding. However, in this method, the tolerance to attacks tends to be weak. Hence, we propose a high-tolerance watermarking method using BCH code which is one of error correcting code. This paper evaluates whether our method preserves the sound quality while ensuring high tolerance.

In this paper, we propose an audio watermarking method using processed sound of music. For audio signals, watermarks are embedded in time domain or Fourier domain. In existing methods, it is required that noises are not perceived in stego signal. On the other hand, in the proposed method, there is no problem even if a different sound from host signal is perceived when stego signal is maintained as music. Accordingly, we use watermark signal which has same timbre as host signal by processing original music data itself. In this paper, we use pitch-transformed sound as processed sound of original music data and we intend to improve the subjective sound quality of stego signal.

In this paper, we propose a new watermarking method based on modification of sound pressure level between microphones. We focus on two characteristics of sound source. One is location of a dominant sound source will be maintained even if signal processing like MP3 compression is performed. Two is deterioration of sound quality is imperceptible even if location of a dominant sound source is modified slightly. By using these characteristics, it is confirmed that we can embed the watermark information with high sound quality and high robustness to MP3 compression. Moreover, we confirm the validity of proposed method using music data having sparse representation. © 2013 IEEE.

The objective of this work is to devise a new way to embed a watermark into digital audio signal. In conventional methods, it is required that the watermark is embedded without noise perception. However, it is difficult to embed the watermark without noise perception. Hence, we propose an embedding method so as to permit a perception of an embedded watermark signal. For watermarked audio signal, high sound quality is needed even if noises are perceived. Therefore, we estimate notes into music data and decide their timbre by musical instrument identification. Furthermore, for improving the sound quality, we embed one bit of watermark into one tone using the corresponding sampled sound for each tone. © 2012 IEEE.

論文番号 D-W1-05

論文番号 F-W1-05

論文番号 F-W2-01

論文番号 F-W2-03

論文番号 F-W2-02

The information hiding technology is classified into two categories: steganography and digital watermark. In the steganography, it is desirable to avoid feeling unnaturalness from stego-data. In the digital watermark, it is also desirable to avoid perceiving the influence of embedding process from stego-data. In the other words, it is important that users do not feel unnaturalness and a sense of incongruity from the stego-data processed by the information hiding technology. The quality of stego-data is often discussed technically by comparing stego-data with cover-data. For example, SNR and ODG are generally used as objective evaluation index in audio signal, and PSNR and SSIM are used in image and movie signal. On the other hand, cover-data are not always shown to users in the practical system applied the information hiding technology; rather, it is thought that cover-data are not shown in many cases. Therefore, for the purpose of getting natural feel from stego-data without unreasonable consideration of the difference between stego-data and cover-data, the audio watermarking method that applied music theory to information hiding is mentioned in this report.

The objective of this work is to propose an embedding method using blind source separation. High sound quality is required even if a watermark is embedded in audio signal. For meeting this requirement, in this paper, we deal with music data not in terms of signal processing but in terms of music theory. For example, it is considered that sound quality is not annoying when a sound having similar timbre included in music data is added under certain conditions. However, if the sound does not become music by adding some sound source, there is something wrong. Hence, in this paper, we focus on consonance in music theory. Moreover, we propose an embedding method having high sound quality by adding watermark signal so that melody line and watermark signal become consonance.

Technical committee on Information Hiding and its Criteria for evaluation (IHC) has established the first criteria for audio watermarking technology. A contest based on the criteria was held on 6 Sep. 2012. Six technologies of audio watermarking, including those which did not satisfy the criteria and/or applied after the deadline, were served for evaluation. The results of the contest shows that no technology was satisfied the criteria in the course of the contest, however, two technologies satisfied the criteria after the deadline. A review on an outline and evaluation of the applied technologies reveals current state and problems of recent audio watermarking technology. More progress in technology of audio data hiding is expected by the succeeding contests based on unified criteria for data hiding.

論文番号3-1-24

論文番号3F1-4

論文番号3F2-2

論文番号3F2-1

論文番号EMM2011-61

論文番号EA2011-100

In our multilayer type audio watermarking method, we increase the capacity of embedding by introducing multi-layered embedding process of watermark information. However, there is the problem that sound quality degradation is likely to be perceived according to multi-layered embedding process. In this research, we reduce the sound quality degradation by introducing "optimized window function" and "sound quality improvement technique using error correcting code" into the multilayer type audio watermarking method. As a consequence, we realized the balance of embedding capacity and sound quality.

In this projeact we propose a system, which generates natural language descriptions of human behavior from image sequences and speech. In general, video-monitoring systems are widely used for pelient monitoring television system in medical facilities, monitoring rare animals in a zoo and condition monitoring for security systems. It is necessary for these systems to reduce traffic under limited communication capacity when transmit the scene of monitoring objects or the condition. The proposed system sequentially estimates the 3-D position and orientation of a man appeared on a monitoring image sequence. Then, the trace or series of estimated positions is divided into concrete motion segments. For each segment, features of an action of the man appeared on the monitoring images is extracted and associated with the most suitable verb, in some case carried with the aid of speech recognition. In addition, some sequences of verbs are combined into high-level verbs. Finally the system composes sentences expressing the movements of the man using the extracted verb and the corresponding frame structure expression based on the flame structure grammar.

本課題では、様々な環境下における音声による対話(マン-マシンインターフェイス)の実現を目的とし、視聴覚融合音声認識システムに関する研究を行なった。具体的には,「人間の目の役割を演ずるテレビカメラ」と「耳の役割を演ずるマイクロホン」から得られた情報を対象としてモバイルコンピュータ上で統合認識処理を行なうことにより、柔軟性のあるマン-マシンインターフェイスの実現を試みている。本研究の目的である「モバイル視聴覚融合音声認識システム」は,3つの処理(「1.視聴覚情報のデータ収集」,「2.融合および認識手法の開発」,「3.モバイルコンピュータへの実装」)から構築される。平成10年度は,「3.モバイルコンピュータへの実装」に関する研究を行なった。また同時に、昨年度までの成果に改良を加えることにより認識精度の改善を行なった。・モバイルコンピュータへの実装昨年度までの研究成果である視聴覚融合音声認識システムをモバイル型マルチメディアコンピュータ上に実装し、室外などの様々な騒音環境下を想定して音声認識実験を行なった。その結果、各種実験により高騒音環境における認識率改善の点で、提案手法が有効であることを確認できた。・融合手法の改善昨年度までの研究成果である視聴覚融合音声認識システムにおける視聴覚融合手法に改善を加えた。具体的には、HMMの出力確率の計算を行なう際にDempster-Shafer確率理論を導入することで、背景騒音などの認識環境により適合した融合を行なうことが可能となり、結果として認識精度の向上を実現した。なお,上述の実験に用いたモバイル型マルチメディアコンピュータは、本科学研究費補助金研究により購入した設備備品である。

本課題では,動画像を使用した視聴覚融合による人間と計算機との対話システムの実現を目的とし,視聴覚融合音声認識システムに関する研究を行なった.特に,近年,急速に普及しつつあるマルチメディアコンピュータ(「動画像の表示」や「音響信号の録音・再生」などの機能を有するコンピュータ)上に視聴覚融合音声認識手法を実装することで,特殊な装置を一切用いることなく音声認識システムを実現することを試みた.本研究で構築を行なった「マルチメディアコンピュータによる視聴覚融合音声認識システム」では,マルチメディアコンピュータが有している“音声同期型動画像入力機能"を用いてフルフレーム(30fps, 1秒間当たり30フレーム)・フルカラー(24 bit color)の動画像をディジタル形式のデータとして撮影する.その後,この動画像を対象として,特徴抽出処理,視聴覚融合処理,音声認識処理の各処理をマルチメディアコンピュータ上で行なう.なお,視聴覚融合処理については,「1次結合によるHMM(隠れマルコフモデル)視聴覚融合手法」と「TDNN(時間遅れニューラルネットワーク)による視聴覚融合手法」の2種類の手法により実現している.本システム用いて音声認識実験を行なった結果,・唇の動きが速いために認識が困難であった子音に対する効果・マルチメディアコンピュータ上への視聴覚融合音声認識システムの実装可能性を確認した.さらに,認識率を改善する方策として,口唇形状をより反映する視覚特徴量の抽出法について検討を進めている.なお,上述のシステムの構築,および,実験評価の実施に際して,本科学研究費補助金研究により購入した設備備品を使用した.

本課題では,動画像を使用した視聴覚融合による人間と計算機との対話システムの実現への第一段階として,比較的発話時間の短い文章(人間から計算機への一方通行)を対象とした視聴覚融合による音声認識システムを実現する事を目的とし,動画像を用いた視聴覚融合音声認識システムに関する研究を行なった.本研究で構築を行なった「フルフレーム画像を対象とした視聴覚融合音声認識システム」では,視聴覚情報の入力手段として “音声同期型動画像入力機能を有しているマルチメディアパソコン" を用いて,フルフレーム(30fps,1秒間当たり30フレーム)の動画像をディジタル形式のデータとして撮影する.その後,この動画像を対象として,HMM(隠れマルコフモデル)に多次元ベクトル量子化を組み合わせた認識モデル上で,特徴抽出処理,視聴覚融合処理,音声認識処理の各処理を行なう.なお,本システムでは,「視覚情報用(動画像用)HMMにより算出された対数尤度」と「聴覚情報用(音声用)HMMにより算出された対数尤度」とを1次結合するというシンプルかつ効果的な手法により視聴覚融合処理を実現している.本システム用いて音声認識実験を行なった結果,・母音発声時の音声認識精度の向上・唇の動きが速いために動画像による認識が困難であった子音に対する効果を確認した.さらに,ニューラルネットワークを利用した視聴覚融合処理方式についても検討を進めており,今後は音声認識システムへの実装を試みる予定である.なお,上述のシステムの構築,および,実験評価の実施に際して,本科学研究費補助金研究により購入した設備備品を使用した.

18面 Paper

Media report

Paper