1 Introduction
Cognitive diagnosis is an essential and fundamental technology in smart education systems, in which cognitive diagnosis can help to obtain the proper profiles of students and assist lots of education services, such as student learning report and adaptive exercise recommendation kuh2011 ; lana . Figure 1 shows an example of cognitive diagnosis process. Generally, with the exercise responses of students and labels of exercises, cognitive diagnosis is to infer their relative abilities irt , such as proficiency on specific knowledge concepts (e.g. multiplication of rational number) ncd .
Many classical methods have been developed to address this issue, such as Multidimensional Item Response Theory (MIRT) mirt , Deterministic Inputs, Noisy And gate model (DINA) dina , Matrix Factorization (MF) mf , and Item response ranking framework irr
. Recently, deep neural network has also been applied in cognitive diagnosis.
ncdproposed a Neural Cognitive Diagnosis framework (NCD) which utilizes a knowledge proficiency vector to represent student and formulates the students, exercises and responses with an MIRTlike multilayer perceptron. In
ecd an Educational contextaware Cognitive Diagnosis framework (ECD) was developed to model the context of student, e.g. highest education degree of parents and duration in early childhood education. However, since only a knowledge proficiency vector is used to represent the student, these methods are short of characterizing the complete profile of student, such as the comprehensive ability of student, or the average mastery on associated knowledge concepts. For Example, as shown in Figure 1, although Tom and Jim both answer incorrectly, it’s still not suitable for the cognitive diagnosis system to give similar poor scores on concept multiplication of rational number, since Tom is obviously a good student and has a better performance in the domain of rational number.In this paper, we develop a structure to enrich the representation of students by making use of the hierarchical relations of knowledge concepts and the embedding of students. First, the proficiency on parent knowledge concepts is used to represent the knowledge related profile of students, and the proficiency on child concepts is obtained with a parentchild concepts projection layer. Second, for a further exploration on the representation of students, we adopt a lowdimension dense vector as the embedding of each student, and obtain the second knowledge proficiency with a full connection layer. Then, we average the two proficiency above to get the final representation of students, and formulate the students, exercises and responses as a neural diagnosis network. Experiments show that it has a substantial improvement in terms of both response prediction and knowledge proficiency diagnosis.
2 Model
2.1 Problem Definition
Suppose in a smart education system there are students and exercises, and define the responses of students as , where , and denote the th student, th exercise and the relative response of student i on exercise j respectively. In addition, we define the Qmatrix (usually labelled by experts) as , in which denote whether exercise relates to the th knowledge concept, and is the number of concepts. Then, given the responses of students and the Qmatrix
, the goal of cognitive diagnosis is to estimate the knowledge proficiency of each student.
2.2 Student Representation
The proposed method of student representation is illustrated in Figure 2(a). We first notice that, the knowledge concepts to be diagnosed have related parent knowledge concepts that are labeled by experts in advance. As the case shown in Figure 1, rational number is the parent knowledge concept of both multiplication of rational number and division of rational number. Generally, the proficiency in parent knowledge concepts can somehow indicate students’ knowledge related profile and mastery in child knowledge concepts.^{1}^{1}1Without specification, the term knowledge concepts or child knowledge concepts denotes the leaf nodes of concept tree, and parent knowledge concepts denotes the parent nodes of leaf nodes. Therefore, the parentchild relations of knowledge concepts can be used to enrich the representation of students. Suppose the knowledge concepts has parent concepts, and we use a trainable vector to represent the proficiency of th student in each parent concepts. Then the knowledge proficiency in child concepts can be obtained by:
(1) 
where
denotes the sigmoid function,
is a bias vector, and
is a parentchild map matrix in which is a trainable variable if the th child concept is descendent of the th parent concept, and otherwise.In the other hand, only utilizes the relations of same knowledge concept family, and cannot fit the proficiency of different concept families, or the memory and comprehensive ability of students. Hence, we use a lowdimension dense vector as the embedding of student , and get the knowledge proficiency by:
(2) 
where is a bias vector, and is a projection matrix to knowledge concepts.
Then, we can get the final knowledge proficiency by simply calculating the mean:
(3) 
Note that, like the work in ecd , one also get by a weighted sum of and , in which the weight is also trainable. However, we do not see it has a substantial improve in our dataset.
2.3 Answer Correctness Prediction
With the knowledge proficiency obtained above, the task of cognitive diagnosis can be formulated as an answer correctness prediction problem ncd . The structure of prediction layer is shown in Figure 2(b). Specifically, for the exercise , we define as the th column of Qmatrix , and as the discrimination and difficulty embedding respectively, and the prediction of answer correctness is obtained by:
(4)  
(5)  
(6)  
(7) 
where denotes a multilayer perceptron, and is the prediction result. Thus the cross entropy loss for student on exercise is defined as:
(8) 
In addition, to satisfy the monotonicity assumption mirt to ensure good performance and interpretability, we restrict in (1) and weight matrix of the multilayer perceptron (7) to be positive when training ncd . Thus, the higher each entry of or is, the more likely the student answers the exercise correctly. Also note that both the knowledge proficiency and can also be passed to (6) independently for prediction (for reason of same dimentions and same representation abilities). As shown in Figure 2(b), for simplification, we denotes the method using the parent knowledge as PKNCD(arent nowledge), method using the student embedding as EMBNCD(edding), and method using the student representation as SRNCD(tudent epresentation) respectively.
3 Experiments
3.1 Datasets, Metrics and Setups
We test the cognitive diagnosis models with two datasets of realworld education scenarios, i.e. ASSIST assist and XCLASSMATH. See the datasets details in Appendix A. Besides, since there are no groundtruth values for the knowledge proficiency of students, it is difficult to evaluate the models straightforwardly. Following the work in irr , we evaluate the performance of models from two perspectives. First, we use Accuracy (ACC) and Area Under the Curve (AUC) to test the classification abilities of models. Second, we adopt Degree Of Agreement (DOA) to assess the monotonicity of models. See the definition of DOA in Appendix B.
We evaluate the proposed EMBNCD, PKNCD and SRNCD defined in Section 2.3 in the experiments. Since ASSIST does not have information about parent knowledge concepts, only EMBNCD is tested in its experiments. Beside, we adopt two hidden layers in the MLP (7), and set the dimension as 512, 256 for ASSIST, and 128, 64 for XCLASSMATH respectively, and the dimension of student embedding is set to . We also compare the performance of proposed methods with several previous works: DINAdina , MIRTmirt , NCDncd .
3.2 Resutls
Model  ASSIST  XCLASSMATH  

ACC  AUC  DOA  ACC  AUC  DOA  
DINA  0.682  0.727  0.603  0.670  0.712  0.629 
MIRT  0.724  0.733  0.601  0.746  0.754  0.632 
NCD  0.726  0.757  0.609  0.745  0.763  0.635 
EMBNCD  0.735  0.771  0.681  0.748  0.768  0.658 
PKNCD        0.753  0.768  0.656 
SRNCD        0.757  0.780  0.664 
The experimental results are shown in Table 1. One can observe that the proposed methods outperform all the other baselines on both datasets. Specifically, even though simply adding a student embedding layer, EMBNCD can obtain a significant improvement compared with the original NCD. Furthermore, PKNCD and EMBNCD have similar performance, and by combining them we can acquire another obvious gain. Thus, the experimental results demonstrate the effectiveness of the proposed student representation methods.
Meanwhile, we also display the distribution histogram of knowledge proficiency obtained from XCLASSMATH in Figure 3. It’s interesting to notice that the knowledge proficiency of MIRT and NCD have almost the same distribution, since MIRT is a special case of NCD ncd . Besides, the distribution curve of DINA, MIRT and NCD are bimodal. On the contrary, SRNCD has a convex and much more smooth curve, which indicates the proficiency acquired might be more discriminative.
4 Conclusion
In this paper, we considered the problem of student representation in cognitive diagnosis model. We developed a method of student representation with the exploration of the hierarchical relations of knowledge concepts and student embedding. Experiments demonstrate the effectiveness and interpretability of the proposed methods.
References
 (1) Kuh G. D., Kinzie J., Buckley J. A., Bridges B. K., and Hayek J. C. (2011) Piecing together the student success puzzle: research, propositions, and recommendations: ASHE Higher Education Report, volume 116. John Wiley & Sons.
 (2) Yuhao Zhou, Xihua Li, Yunbo Cao, Xuemin Zhao, Qing Ye, and Jiancheng Lv (2021) LANA: towards personalized deep knowledge tracing through distinguishable interactive sequences. In Proceedings of the Educational Data Mining.
 (3) Embretson S. E., and Reise S. P. (2013) Item response theory. Psychology Press.
 (4) Reckase, M. D. (2009) Multidimensional item response theory models. In Multidimensional Item Response Theory, pp. 79112. Springer.
 (5) De La Torre, J. (2009) Dina model and parameter estimation: A didactic. Journal of educational and behavioral statistics 34(1):115–130.
 (6) Koren Y., Bell R., and Volinsky C. (2009) Matrix factorization techniques for recommender systems. Computer 42(8), pp. 30–37.

(7)
Shiwei Tong, Qi Liu, Runlong Yu, Wei Huang, Zhenya Huang, Zachary A. Pardos, and Weijie Jiang (2021) Item response ranking for cognitive diagnosis. In
Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence
, pp. 17501756.  (8) Fei Wang, Qi Liu, Enhong Chen, Zhenya Huang, Yuying Chen, Yu Yin, Zai Huang, and Shijin Wang (2020) Neural cognitive diagnosis for intelligent education systems. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 6153–6161.
 (9) Yuqiang Zhou, Qi Liu, Jinze Wu, Fei Wang, Zhenya Huang (2021) Modeling contextaware features for cognitive diagnosis in student learning. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 2420–2428.
 (10) Feng M., Heffernan N., and Koedinger K. (2009) Addressing the assessment challenge with an online system that tutors as it assesses. User Modeling and UserAdapted Interaction 19(3): 243266.
Appendix A Datasets
Statistics  ASSIST  XCLASSMATH 

# students  4,163  165 
# exercises  17,746  250 
# knowledge concepts  123  74 
# parent knowledge concepts  /  7 
# response logs  324,572  13,574 
# average logs per student  77.97  82.27 
# average logs per exercise  18.29  54.30 
The statistics of the datasets are summarized in Table 2. ASSIST (ASSISTments 20092010 “skill builder”) is a widely used open dataset collected by the ASSISTments online tutoring systems^{2}^{2}2https://sites.google.com/site/assistmentsdata/home/assistment20092010data/skillbuilderdata20092010. XCLASSMATH is a mathematical dataset collected by the smart education system XCLASS^{3}^{3}3https://xclass.qq.com. XCLASSMATH will be released later.
, in which teachers assign and correct students’ homework online, and students do their homework with an eink pad. It mainly contains the mathematical homework logs within two months of the
th grade students of an middle school.Appendix B Degree of Agreement
The Degree of Agreement (DOA) is defined as:
(9) 
where , denotes the proficiency of student on concept , if and otherwise, if exercise contains concept and otherwise, and if both student and did exercise and otherwise. The perspective of DOA is that, if student has a higher proficiency on concept than student , then student is more likely to answer exercise related to concept correctly than student . The average of on all concepts is used in our experiments. Thus, the model with a higher DOA score might have a better monotonicity and interpretability.
Comments
There are no comments yet.