Abstract Perception of facial expression is crucial for primate social interactions. This visual information is processed through the ventral cortical pathway and the subcortical pathway. However, the subcortical pathway exhibits inaccurate processing, and the responsible architectural and physiological properties remain unclear. To investigate this, we constructed and examined convolutional neural networks with three key properties of the subcortical pathway: a shallow layer architecture, concentric receptive fields at the initial processing stage, and a greater degree of spatial pooling. These neural networks achieved modest accuracy in classifying facial expressions. By replacing these properties, individually or in combination, with corresponding cortical features, performance gradually improved. Similar to amygdala neurons, some units in the final processing layer exhibited sensitivity to retina-based spatial frequencies (SFs), while others were sensitive to object-based SFs. Replacement of any of these properties affected the coordinates of the SF encoding. Therefore, all three properties limit the accuracy of facial expression information and are essential for determining the SF representation coordinate. These findings characterize the role of the subcortical computational processes in facial expression recognition.

Cultural similarities and differences in facial expressions have been a controversial issue in the field of facial communications. A key step in addressing the debate regarding the cultural dependency of emotional expression (and perception) is to characterize the visual features of specific facial expressions in individual cultures. Here we developed an image analysis framework for this purpose using convolutional neural networks (CNNs) that through training learned visual features critical for classification. We analyzed photographs of facial expressions derived from two databases, each developed in a different country (Sweden and Japan), in which corresponding emotion labels were available. While the CNNs reached high rates of correct results that were far above chance after training with each database, they showed many misclassifications when they analyzed faces from the database that was not used for training. These results suggest that facial features useful for classifying facial expressions differed between the databases. The selectivity of computational units in the CNNs to action units (AUs) of the face varied across the facial expressions. Importantly, the AU selectivity often differed drastically between the CNNs trained with the different databases. Similarity and dissimilarity of these tuning profiles partly explained the pattern of misclassifications, suggesting that the AUs are important for characterizing the facial features and differ between the two countries. The AU tuning profiles, especially those reduced by principal component analysis, are compact summaries useful for comparisons across different databases, and thus might advance our understanding of universality vs. specificity of facial expressions across cultures.

Schizophrenia affects various aspects of cognitive and behavioural functioning. Eye movement abnormalities are commonly observed in patients with schizophrenia (SZs). Here we examined whether such abnormalities reflect an anomaly in inhibition of return (IOR), the mechanism that inhibits orienting to previously fixated or attended locations. We analyzed spatiotemporal patterns of eye movement during free-viewing of visual images including natural scenes, geometrical patterns, and pseudorandom noise in SZs and healthy control participants (HCs). SZs made saccades to previously fixated locations more frequently than HCs. The time lapse from the preceding saccade was longer for return saccades than for forward saccades in both SZs and HCs, but the difference was smaller in SZs. SZs explored a smaller area than HCs. Generalized linear mixed-effect model analysis indicated that the frequent return saccades served to confine SZs’ visual exploration to localized regions. The higher probability of return saccades in SZs was related to cognitive decline after disease onset but not to the dose of prescribed antipsychotics. We conclude that SZs exhibited attenuated IOR under free-viewing conditions, which led to restricted scene scanning. IOR attenuation will be a useful clue for detecting impairment in attention/orienting control and accompanying cognitive decline in schizophrenia.

OBJECTIVE: In long-term video-monitoring, automatic seizure detection holds great promise as a means to reduce the workload of the epileptologist. A convolutional neural network (CNN) designed to process images of EEG plots demonstrated high performance for seizure detection, but still has room for reducing the false-positive alarm rate. METHODS: We combined a CNN that processed images of EEG plots with patient-specific autoencoders (AE) of EEG signals to reduce the false alarms during seizure detection. The AE automatically logged abnormalities, i.e., both seizures and artifacts. Based on seizure logs compiled by expert epileptologists and errors made by AE, we constructed a CNN with 3 output classes: seizure, non-seizure-but-abnormal, and non-seizure. The accumulative measure of number of consecutive seizure labels was used to issue a seizure alarm. RESULTS: The second-by-second classification performance of AE-CNN was comparable to that of the original CNN. False-positive seizure labels in AE-CNN were more likely interleaved with "non-seizure-but-abnormal" labels than with true-positive seizure labels. Consequently, "non-seizure-but-abnormal" labels interrupted runs of false-positive seizure labels before triggering an alarm. The median false alarm rate with the AE-CNN was reduced to 0.034 h-1, which was one-fifth of that of the original CNN (0.17 h-1). CONCLUSIONS: A label of "non-seizure-but-abnormal" offers practical benefits for seizure detection. The modification of a CNN with an AE is worth considering because AEs can automatically assign "non-seizure-but-abnormal" labels in an unsupervised manner with no additional demands on the time of the epileptologist.

Identification of genotypes is crucial for treatment of glioma. Here, we developed a method to predict tumor genotypes using a pretrained convolutional neural network (CNN) from magnetic resonance (MR) images and compared the accuracy to that of a diagnosis based on conventional radiomic features and patient age. Multisite preoperative MR images of 164 patients with grade II/III glioma were grouped by IDH and TERT promoter (pTERT) mutations as follows: (1) IDH wild type, (2) IDH and pTERT co-mutations, (3) IDH mutant and pTERT wild type. We applied a CNN (AlexNet) to four types of MR sequence and obtained the CNN texture features to classify the groups with a linear support vector machine. The classification was also performed using conventional radiomic features and/or patient age. Using all features, we succeeded in classifying patients with an accuracy of 63.1%, which was significantly higher than the accuracy obtained from using either the radiomic features or patient age alone. In particular, prediction of the pTERT mutation was significantly improved by the CNN texture features. In conclusion, the pretrained CNN texture features capture the information of IDH and TERT genotypes in grade II/III gliomas better than the conventional radiomic features.

INTRODUCTION: Epileptologists could benefit from a diagnosis support system that automatically detects seizures because visual inspection of long-term electroencephalograms (EEGs) is extremely time-consuming. However, the diversity of seizures among patients makes it difficult to develop universal features that are applicable for automatic seizure detection in all cases, and the rarity of seizures results in a lack of sufficient training data for classifiers. METHODS: To overcome these problems, we utilized an autoencoder (AE), which is often used for anomaly detection in the field of machine learning, to perform seizure detection. We hypothesized that multichannel EEG signals are compressible by AE owing to their spatio-temporal coupling and that the AE should be able to detect seizures as anomalous events from an interictal EEG. RESULTS: Through experiments, we found that the AE error was able to classify seizure and nonseizure states with a sensitivity of 100% in 22 out of 24 available test subjects and that the AE was better than the commercially available software BESA and Persyst for half of the test subjects. CONCLUSIONS: These results suggest that the AE error is a feasible candidate for a universal seizure detection feature.

Binocular rivalry is a phenomenon created by presenting similar but different images for both eyes simultaneously. Many previous studies have investigated various brain responses to binocular rivalry. However, a response of the perceptual transition in binocular rivalry has not been clear yet. The present study aimed to measure the response of the perceptual transition in binocular rivalry using a motion rivalry stimuli with various motion angles. It is known that the perception of motion rivalry stimuli has two conditions depending on the angle between two motion directions. One is a rivalrous condition that cause binocular rivalry and the perceptual transition, and the other is a fused condition that does not cause them. Visual evoked fields (VEFs) were recorded with five healthy subjects using a 440-channel whole-head magnetoencephalogram (MEG) system. We classified trials to rivalrous or fused conditions, and calculated time averages of root mean square (RMS) values for every 100 ms in each condition. As a result, the time average of RMS values of the rivalrous condition were significantly larger than those of the fused condition after 400 ms post-stimulus. These results suggested that the perceptual transition in binocular rivalry increased the late MEG component.

We reported previously that human coagulation factor IX (F-IX), when activated by normal human red blood cells (RBCs). causes coagulation. We also identified and characterized the F-IX-activating enzyme in the normal RBC membrane. In the present study, the coagulation of blood in experimental animals, including swine, dogs, rabbits, cattle and sheep, was compared to that in humans, with special reference to the procoagulant activity of RBCs. Rheological measurement showed that coagulation of platelet-free plasma (PFP) in a polypropylene tube did not occur in any of the species. In swine as in humans, coagulation of PFP supplemented with RBCs (RBCs/PFP) occurred. However, in dogs, rabbits, sheep or cattle coagulation of RBCs/PFP did not occur. Fluorescence assays of RBC membranes using a synthetic fluorogenic substrate suggested that F-IX-activating enzyme may be present in swine, dog and rabbit as well as human RBC membranes, but its level may be very low in sheep and bovine membranes. Our data suggest that there is a significant difference in procoagulant activity of RBCs among animal species. In addition, they suggest that appropriate selection of animal species would be important for studying venous thrombus formation, including the evaluation of anticoagulability of materials under stagnant flow conditions.

本研究は，人間の視覚系を模倣した大規模な計算モデルである深層ニューラルネットワーク（Deep Neural Network: DNN）を用いて人間の視覚系と同様の情報処理を学習させたときに，DNNが人間の視覚と同じように錯視（形，色，動きなどが本来とは違う現象）を生じるかを確認することで，DNNが人間の視覚系を研究する上での計算機モデルとして適切であるかを評価することを目的とする． R3年度の研究では，まず深層ニューラルネットワーク(DNN)を用いた視覚研究を実施するためのシステムを構築した．具体的には，高速な繰り返し計算に用いる画像処理装置（GPU）を搭載したコンピュータを調達し，その中に DNN を実装するためのシステムを導入した．手始めに，実験を担当する学生の卒業研究として視覚研究に基づいて構成された動画処理用の DNN である PredNet を用いた錯視画像（「蛇の回転」）の情報処理に関する研究を行なった．この DNN を用いた研究では，先行研究（Watanabe et al., 2018）において用いられたものと同じ Chainer を用いて実装した深層学習プログラムを用い，学習させる動画像セットに工夫を施して学習効率の変化を調べることにより，学習されている画像特徴の推定を試みた．この研究は学部４年の卒業研究として実施され，学生は卒業論文を提出して卒業し，大学院へ進学した．この知見は視覚メカニズムを研究するための計算モデルとして深層学習を評価する上で重要であり，R4年度に国内の学会にて成果発表を行う予定である．この研究を今後も発展させつつ，本題である#TheDress画像の問題を対象とした DNN の研究を推進していく．

生体の脳のような高いエネルギー効率をもつ情報処理システムの実現を目標に、脳の高効率性の要因の一つであると考えられる、情報のゲーティングを明らかにするために、神経集団における集団発火の伝播のシミュレーションおよびにその解析を行った。前年度までに進めてきたFokker-Planck方程式による定式化を用いて、Synfire Chainと呼ばれる神経細胞集団中での同期発火の伝播モデルにおいて、一般に用いられている線形なLeaky Integrate-and-Fire (LIF) モデルと、Naイオン電流の項を持つ非線形なExponential Integrate-and-Fire (EIF) モデルの比較を行った。その結果、自発発火を起こし、Naイオン電流が活性化されるような条件下では、EIFモデルの膜電位分布が広がり、非同期な状態となることが示された。この非同期状態は、自発発火の存在に強く依存するため、微弱な抑制性入力による弱い過分極によって容易に消滅する。このことは、微弱な抑制性入力によって神経集団の膜電位の同期状態が制御可能であり、集団発火の伝播のゲーティングが可能となることを示唆している。これらの研究結果は2021年の北米神経科学会の年大会において発表された。本研究をさらに推進することによって、環境ノイズをうまく利用しつつ脳のように高いエネルギー効率での情報処理を可能とするシステムや、脳における注意のメカニズムなどの新しい知見が得られることが期待される。

We investigated and established a method to measure the distribution of visual attention by using visually evoked potential (VEP). When an image of visual scene is sectored and each sector flickers in time, VEPs with corresponding temporal frequency (SSVEP) will be induced. By monitoring the changes in amplitude of this SSVEP we can monitor the state of visual attention at each sector. In the present study, we sectored left/right eye images in different ways; e.g., left-eye image was sectored vertically and right-eye image was sectored horizontally. If the number of vertical and horizontal sectors are 5 and 3, respectively, the total number of frequency used is 8 but we can obtain attentional state of 15 sectors. According to the reduction of SSVEP frequencies, it was possible to obtain better S/N ratio, and we succeeded in estimating the focus of attention by this method.

We studied brain responses under binocular rivalry to clarify the temporal dynamics of visual perception. Binocular rivalry is a visual phenomena which causes temporally random perceptual alternation by presenting two different images to each eye. Since recording of brain responses usually requires several tens of iterations, and is not suitable to record such a temporally random responses, we developed a new method called 'phase template analysis'. The new method enabled a one-shot recording of the brain responses under binocular rivalry, and clarified the temporal process of the visual perception. The method also applied to develop a brain machine interface (BMI), resulting a biped robot controlled by brain responses recording with electroencephalogram (EEG).