scimat2
None
view repo
In this work, we announce a comprehensive well curated and opensource dataset with millions of samples for precollege and college level problems in mathematicsand science. A preliminary set of results using transformer architecture with character to character encoding is shown. The dataset identifies some challenging problem and invites research on better architecture search
READ FULL TEXT VIEW PDFNone
Datasets play an important role in driving research in supervised machine learning research. Some prominant examples being MNIST
lecunmnisthandwrittendigit2010 for hand written digit classification, CIFAR10 cifar10and IMAGENET
imagenet for image classification and generative models, etc. For solving math word problems, semantic rules and various models have been proposed in NLP community since 1963 starting from bobrow , briars1984 , feigenbaum1963computers . Some word problems in fletcher1985 are of the form: Lucy has two dimes. Sarah has six dimes. How many dimes do they have altogether? or Dan has six books. Jill has two books. How many books does Dan have more than Jill? This paper uses Kintsch and Greeno’s (1985) theory of comprehension and solution for arithmetic word problems above. These papers used classical approaches with semantic rules. Recently, machine learning models have been used for which large labelled dataset is essential. Hence, there is a dire need of large questionanswer dataset for mathematics and science problems; such dataset can have impact on online education, intelligent tutoring and automated grading. For intelligent tutoring, not just the answers, but the step by step hint can be provided; this is explored in kang2016. However, tutoring requires some knowledge graph representation. Although, this was shown for simple algebraic and geometric mathematics problems, it remains a challenging task for more advanced problems. No wonder tutoring is a complex task as nicely pointed out in detail in
kenneth2013 . Given that intelligent tutoring is one of the most challenging task, the datasets and innovative architectures would play a critical role to succeed in this endevour. Recently, question answer dataset^{2}^{2}2https://github.com/deepmind/mathematics_dataset for mathematics was proposed in AnalysingMR and for word problem sample dataset was proposed in SMWPSDL1 , and a comparison of results for character to character encoding for transformer and for LSTM is shown. This dataset has selected problems in mathematics for math exams for British 16 year old school children. Some sample questions are: or Three letters picked without replacement from qqqkkklkqkkk. Give prob of sequence qql. In clark2018think , a set of 7787 multiple choice questions in high school science questions is proposed as ARC (AI2 Reasoning challenge, 2018). A sample question from this dataset is: Which property of a mineral can be determined just by looking at it? (A) luster [correct] (B) mass (C) weight (D) hardness. Moreover, with the ARC challenge a large corpus of 14 million science sentences relevant to the questionanswer set is also proposed. A sample sentence from the corpus is: Random motion of the air molecules and turbulence provide upward forces that may counteract the downward force of gravity. Such a corpus allows language understanding and questions with linguistic variations. We remark that any other corpus can be used for training the given architecture for linguistic understandings, which is further trained on the given datasets. For the ARC challenge, several baseline neural models were proposed. There are datasets for logical reasoning and English comprehension. For example, in weston2015aicomplete , logical reasoning question answer dataset is proposed. The reasoning is considered to be of various types such as problems involving single supporting fact, two supporting fact, counting, path finding, size reasoning, etc. A sample question for path finding is: The kitchen is north of the hallway. John is hungry. The bathroom is west of the bedroom. John goes to the kitchen. The den is east of the hallway. John grabbed the apple there. The office is south of the bedroom. Daniel is hungry. How do you go from den to kitchen? How do you go from office to bathroom?. The last two sentences are questions with answers west, north and north, west respectively. This dataset is part of bAbI project^{3}^{3}3https://github.com/facebookarchive/bAbItasks of facebook research. For algebra word problems, a dataset^{4}^{4}4http://groups.csail.mit.edu/rbg/code/wordprobs/ and code is proposed in kushmanetal2014learning . Most of these word problems correspond to solving system of linear equations, their method derives these equations, then solves it. A sample question answer in this dataset taken from kushmanetal2014learning is: An amusement park sells 2 kinds of tickets. Tickets for children cost $1.50. Adult tickets cost $4. On a certain day, 278 people entered the park. On that same day the admission fees collected totaled $792. How many children were admitted on that day? How many adults were admitted? with soutions x = 128, y = 150. Continuing along these lines in wangetal2017deep, they propose to translate the math word problem to equation using recurrent neural network (RNN) without doing any complex feature extractions. To the best of our knowledge, a comprehensive
opensource dataset for mathematics and science for precollege and college level have been missing. To this end, in the following, we announce a new large dataset named SCIMAT, and we show preliminary results and comparisons.We announce a large dataset^{5}^{5}5https://github.com/misterpawan/scimat2 of hundreds of millions of questionanswer for mathematics and science for precollege and college level, which typically is taught to 1519 age group around the world. The list of topics covered are: Acids And Bases, Atomic Structure, Stoichiometry, Thermodynamics, Units And Dimensions, Kinematics, Laws of Motion, Work Power Energy, Rotatory Motion, Gravitation, Electricity, Moving Charges and Magnetism, Electro Magnetic Induction, Alternating Current, Electro Magnetic Waves, Ray Optics and Optical Instruments, Wave Optics, Dual Nature of Matter, Mechanical Properties of Solids and Liquids, Thermal Properties of Matter, Kinetic theory of Gases, Sound, Waves And Oscillations, SemiConductors, Communication Systems, etc. Each topic contains several subtopics, where each subtopics has hundreds of thousands of question answer dataset.
Question: 33 mL of a solution of HNO3 is found to be completely neutralised by 45 mL of a given solution of NaOH. If we take 12 mL of the same solution of HNO3, the amount of NaOH solution (the same solution as before) required to neutralise it will be. Answer: 16.36 ml
Question: If a diatomic gas of 1 moles at 68 atm and volume 68 lit is adiabatically changed to volume 188 lit, then what will be the pressure. Answer : 16.4atm
Question: A body is dropped from a height of 9578 m with an initial velocity of 42 m/s. With what velocity will it strike the ground ? Answer: 435.3 m/s
Question: A 9062 N force is applied on a body of mass 980 kg placed on a smooth surface, then what is the resulting acceleration obtained ? Answer: 9.2 m/s2
Question: The volume of 549 g of a substance is 116 cm3. If the density of liquid in which substance is placed is 4 g/cm3, will the substance float or sink ? Answer: sink
Question: If a 822 V battery is connected across an unknown resistor, there is 224 A in the circuit, find the value of resistance of the resistor ? Answer: 3.7 ohm
Question: A square coil of side 3 cm consists of 31 turns and carries a current of 5 A. The coil is suspended vertically and the normal to the plane of the coil makes an angle of 53 degress with the direction of a uniform horizontal magnetic field of magnitude 17 tesla. What is the magnitude of the torque experienced by the coil. Answer: 1.9 newtonm
Question: A series LCR circuit is connected to a variable frequency 230 V source with L = 193 H, C = 72 muF, R = 176 ohm. Determine the rms potential drop across resistance? Answer: 230 volt
Question: Suppose that the electric field amplitude of an electromagnetic wave is E0 = 1936 N/C and that its frequency is v = 1512 MHz. Find an expression for B? Answer: 6.45e06sin[3.17e+01x9.50e+09t]
Question: During blood transfusion, the needle is inserted in a vein where the gauge pressure is 1720 Pa. If the blood container is placed at 177 mm above the earth level so that blood may just enter the vein, is it safe for the patient?. Answer: yes, patient is safe
Question: A sound wave travels at a speed of 29980.8 m/s, if it’s wavelength is 32 m, will the sound wave be audible ? Answer: audible
Question: For an amplitude modulated wave, the maximum amplitude is found to be 18.62 V while the minimum amplitude is found to be 7.91 V. Determine the modulation index. Answer: 0.4
Similarly, for mathematics, we append datasets from calculus (differentiation and integration), linear algebra (rank, row reduced echelon form, determinant, trace, etc), set operations, statistics, number theory, probability, etc. Some sample questions in this dataset are the following:
Question: Differentiate 293 * x * (sin(x) + sec(x)) with respect to x
Answer: 293 * x * (cos(x) + tan(x) * sec(x)) + 293 * sin(x) + 293 * sec(x)
Question: Integrate cot(4*x ^{∧}2) + sec(22*x^{∧}2) with respect to x
Answer: 8*x*( cot(4 * x^{∧}2 ) ^{∧}2  1) + 44*x* tan(22*x^{∧}2 ) * sec(22*x^{∧}2 )
Question: 2 * f ( x ) + 8 * Derivative ( f ( x ) , x ) + Derivative ( f ( x ) , ( x, 2 ) ) = 0
Answer: f ( x ) = ( C1 * exp ( x * ( 1  sqrt ( 6 ) ) ) + C2 * exp ( x * ( 1 + sqrt ( 6 ) ) ) )
Question: Calculate the Rank of Matrix [ [2, 1, 3, 7] , [1, 0, 4, 2 ] , [ 3, 1, 7, 9 ] ] Answer: 2
Question: Calculate the Trace of Matrix [ [ 13, 38, 61 ] , [ 29, 1, 39 ] , [ 92, 16, 45 ] ] Answer: 59
Question: What is the union of { 2, 6, 7, 8, 9 } with { 3, 7, 8 } Answer: { 2, 3, 6, 7, 8, 9 }
Question: What is the median of the sequence ( 20, 38, 4, 21, 31, 94, 55) Answer: 31
Question: What is 2 (base 3) in base 8? Answer: 2
Question: Expand (s + s + 2*s**5)*(4  1  2)  3*s**5 + 4*s**5 + 0*s**5  2*s**5  s**5 + 5*s**5 + (3*s**2  4 + 4)*(5*s**3  5*s**3  s**3). Answer: 2*s**5
Question: Three letters picked without replacement from a: 3, c: 1, b: 7, d: 3. Give prob of sequence bdc. Answer: 1/104
Type of problem 



Differentiation of sum  99%  
Differentiation of product  100%  
Differentiation of composition  100%  
Integration of sum  100%  
Integration of product  100%  
Integration of composition  92.5%  
Addition of matrices  49%  
Subtraction of matrices  74%  
Transpose of matrix  100%  
Determinant of matrix  32%  
Multiplication of matrices  32%  
Trace of a matrix  100%  
Product of matrix with a Scalar  100%  
Row Reduced echelon  76%  
Rank of a matrix  92.5%  
Mean of a sequence  95%  
Variance of a sequence  39.5%  
Median of a sequence  99%  
Set Union  100%  
Set intersection  97.5%  
Set difference  100%  
Symmetric difference between sets  100% 
Type of problem 



Neutralization  82.6%  
Adiabatic  76.3%  
Refrigrator  82.6%  
Estimated value  61.8%  
Force, mass, acceleration  45.5%  
Momentun conservation  78.7%  
Kinetic energy  73.5%  
Balancing a metre stick  81.5%  
Gravitational field  94.2%  
Float or sink?  98%  
Ohms Law  89.0%  
Torque due to magnetic field  84.5%  
LCR circuit  91.3%  
Mirror formula for concave  79.6%  
Is the sound audible?  76%  
Sound wave propogation  31.5%  
modulation index  89.6%  
Force between wires  75.2%  
Conservation of momentum  14.5%  
Potential energy  63%  
Work, mass, velocity  10% 
The code for training and testing is written in Python and PyTorch framework is used. The models are trained on dual Intel Xeon E52640 v4 processors, providing 40 virtual cores per node, 128 GB of 2400MT/s DDR4 ECC RAM and four Nvidia GeForce GTX 1080 TiGPUs, providing 14336 CUDA cores. We use the standard transformer described in vaswani2017 with our own specifications as follows. We use an encoder which is composed of stack of identical layers. The embedding size (dmodel) = 128, attention heads (
) = 8. The inner layer size of feed forward network used in each layers of encoder stack (dff) = 512. We minimize the sum of log probabilities of the correct tokens via the Adam optimizer with adaptive learning rate. The model was trained for 100 epochs. For floating point answers, accuracy for two digits after decimal place was matched. In Table
2, 2, we find that there are datasets where it is challenging to obtain high accuracy, and robust architecture or encoding is required. Since lately, many other variants of transformers were proposed, in Table 3, we compare various different transformer with wordtoword and chartochar encoding. In general, we found that char2char gives best accuracy.Type of problem 





pH  99.8%  97.3%  84.7%  
Compare number of atoms  97.5%  94.1%  94.4%  
Operations with significant digits  80.4%  72.9%  67.4%  
Equation of motion  8.5%  12.8%  12.2%  
Kinetic energy  73.5%  72.4%  71.8%  
Float or sink?  98%  98.7%  98.8%  
Series/Parallel combination of resistance  88.9%  32.5%  45.7% 
This work was done at IIIT, Hyderabad. The authors acknowledge all the support of the institute.
Simple mathematical word problems solving with deep learning.
Technical report, Stanford University, 01 2010.Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
, pages 845–854, Copenhagen, Denmark, September 2017. ACL.