Faculty: Randall Davies

Nyland, R., Davies, R., Chapman, J., & Allen, G. (2016). Transaction-Level Learning Analytics in Online Authentic Assessments. Journal of Computing in Higher Education, 1-17. http://dx.doi.org/10.1007/s12528-016-9122-0 http://link.springer.com/article/10.1007/s12528-016-9122-0

For students pursuing engineering degrees, training in ethics, social responsibility, and allied topics is advocated by a variety of stakeholders, mandated in ABET accreditation requirements, and largely presumed by professional associations and licensing bodies. As a consequence, almost all engineering students have at least some exposure to engineering ethics training. Additionally, many formal courses and programs have been created to promote ethical integrity and professional responsibility among engineering graduates, while a variety of other interventions (e.g., service learning programs) have been developed to more broadly challenge engineering students to see themselves as socially engaged citizens and professionals. Nonetheless, there has been a surprising lack of research on development of social and ethical responsibility among undergraduate engineering students. Few studies have systematically examined levels of ethical knowledge, decision-making capabilities, and commitments to social responsibility among large numbers of engineering students, much less examined how such indicators change over time and are impacted (or not) by specific kinds of learning experiences. As a result, faculty, administrators, and other stakeholders have little evidence to guide creation of high-impact courses and programs. Still other recent research suggests that such impacts may be blunted by a “culture of disengagement” that pervades many engineering schools. This paper reports on an National Science Foundation (NSF) supported CCE STEM research project that aims to address some of these gaps. Our study addresses two main questions: • RQ1: What do engineering students perceive as responsible (and irresponsible) professional conduct, and what do they perceive as socially just (and unjust) technical practices?, and • RQ2: How do foundational measures and understandings of social and ethical responsibility change during a four-year engineering degree program, both in general and in relation to specific kinds of learning experiences? This paper offers an overview of the longitudinal mixed-methods study design we are using to collect and analyze quantitative and qualitative data from undergraduate engineering students at four universities. These schools represent a variety of institution types, and each has students in programs of particular interest, e.g., those involving service-learning or intensified ethics instruction. Our data collection efforts were initiated in early Fall 2015, with survey responses collected from 757 first-semester engineering students at four schools. The survey includes items and measures related to engineering ethics knowledge, justice beliefs, political and social involvement, macro-ethical considerations, moral attentiveness, moral disengagement, ethical climate, and extensive demographics. Repeat measures will be collected from as many of the original respondents as possible during their fifth and eighth academic semesters. To more deeply probe constructs and themes of interest, we will also conduct semi-structured interviews with more than 10% of survey respondents during their first and eighth academic semesters. In addition to describing our study design, this paper reports preliminary insights from the first phase of our study. We especially focus on some highlights from our initial analysis of the survey data, as well as how select survey results are being used to stratify the interview sample and tune the interview protocol to focus on some observed patterns in the survey data. We expect this paper will be of interest to scholars involved with teaching and/or conducting research on ethics, social responsibility, and related topics.

As global economies and interactions increase, there is a developing need for students to have a variety of cross-cultural experiences. Traditionally these experiences have occurred through study abroad experiences and internships. With the advent of virtual technologies global virtual teams are an opportunity to provide students with a cross-cultural experience. However, stakeholders need to be aware of their roles and responsibilities in such an endeavor. This article discusses findings from a three-year study seeking to integrate global virtual teams into regular classrooms. The roles and responsibilities of institutions, faculty, and students are discussed in this article.

This study tested an iPad application using a dichotomous key as a scaffolding tool to help students make more detailed observations as they identified various species of birds on display in a museum of natural science. The Mobile Dichotomous Key (MDK) iPad application was used by groups of fifth and seventh grade students. Analysis of the findings suggest that the MDK was equally as effective and in some ways more effective than the educator-led intervention at improving students’ scientific observation skills. In general, the scaffolding key in both mobile and educator-led form was found to be more effective for fifth grade students than for seventh grade students. In addition, the it was found to be effective at improving the level of detail students provided and the number of scientific terms they used, but not at improving the number of valid inferences students made.

The purpose of this research was to explore how technology can be used to teach technological skills and to determine what benefit flipping the classroom might have for students taking an introductory-level college course on spreadsheets in terms of student achievement and satisfaction with the class. A pretest posttest quasi-experimental mixed methods design was utilized to determine any differences in student achievement that might be associated with the instructional approach being used. In addition, the scalability of each approach was evaluated along with students' perceptions of these approaches to determine the affect each intervention might have on a student's motivation to learn. The simulation-based instruction tested in this study was found to be an extremely scalable solution but less effective than the regular classroom and flipped classroom approaches in terms of student learning. While students did demonstrate learning gains, the process focus of the simulation's instruction and assessments frustrated students and decreased their motivation to learn. Students' attitudes towards the topic, their willingness to refer the course to others, and the likelihood that they would take another course like this were considerably lower than those of students in the flipped or regular classroom situations. The results of this study support the conclusion that a technology enhanced flipped classroom was both effective and scalable; it better facilitated learning than the simulation-based training and students found this approach to be more motivating in that it allowed for greater differentiation of instruction.

Federal legislation currently mandates that technology be integrated into school curricula because of the popular belief that through technology learning is enhanced. The challenge for educators is to understand how best to teach with technology while developing the technological expertise of their students. This paper outlines a framework of technological literacy designed to helps educators understand, evaluate, and promote technology integration properly.

Most people agree that young children benefit academically from attending kindergarten; however, some research suggests that any academic advantage attributed to attending full-day kindergarten compared to half-day programs will disappear as early as third grade. Based on the results of this study which looked at students in typical classroom settings, a likely explanation for this is that teachers often provide considerable remedial assistance to low achieving students. Once students meet benchmark expectations, instruction tends to broaden to other instructional objectives. We also found that in practice distinguishing between full- and half-day programs based solely on time in school is problematic. Many schools offering full-day kindergarten do not provide a full day of academic instruction, and many teachers see little value of doing so.

This article explores and summarizes trends in research and scholarship over the last decade (i.e., 1998-2007) for students completing dissertations and theses in the area of distance education. The topics addressed, research designs utilized, and data collection and analysis methods used were compiled and analyzed. Results from this study indicate that most of the distance education research conducted by graduate students in this period of time has been descriptive, often addressing the perceptions, concerns, and satisfaction of various stakeholders with a particular distance education experience. Studies of this type typically used self-report surveys and analyzed the data using descriptive statistics. Validating the concern of many distance education scholars, there was a lack of graduate student research aimed at developing a theory base in distance education. On a positive note, projects directly comparing distance education with traditional face-to-face classrooms to determine the merit of specific programs declined significantly in 2007 compared to 1998. This result might indicate that distance learning is becoming accepted as a viable and important educational experience in its own right. Another encouraging finding was the decreased emphasis on studies focused on technology issues, such as those analyzing the quality of distance education technology and questioning educators’ ability to provide an acceptable technology-enabled distance learning experience.

Creating surveys is a common task in evaluation research; however, designing a survey instrument to gather average group response data that can be interpreted in a meaningful way over time can be challenging. When surveying groups of people for the purpose of longitudinal analysis, the reliability of the result is often determined by the response scale used. Likert scales and numeric continua are commonly used for this purpose. Both scales are relatively easy to create and both have distinct advantages and disadvantages. This paper reports the findings of a project to combine a fully-anchored Likert scale with a numeric rating scale. The resulting scale for a follow-up survey utilizes fixed anchor points along with visual cues to provide a layer of direction for the respondents; it also uses a numeric continuum to elicit a second layer of information that is more discriminating. The results are both easily interpreted on the original scale and are more appropriate for longitudinal comparison purposes both in terms of obtaining reliable results and in the scales ability to identify changes in response over time.

This paper considers the underlying assumptions related to the use of random assignment in educational research and evaluation; more specifically, the ability of random assignment to create similar comparison groups for the purpose of determining the effectiveness of educational programs. In theory, randomly assigning individuals to comparison groups is considered to be the best method available to maximize the likelihood that groups used in this type of research will be similar; however, in educational research designed to identify proven best practices, random assignment of individuals is rarely possible; other methods including random assignment of intact units and non-random selection techniques are often used. Using a database simulation, this study set out to determine the degree to which various selection methods might be effective at creating comparable groups. Given the complex dynamics of the teaching and learning process and the abundance of potentially confounding variables, it seems likely that comparison groups will always be dissimilar to some degree. While random assignment of individuals performed as expected when controlling for a single extraneous factor, the likelihood that comparison groups created in this manner will differ on one or more potentially confounding variables is extremely likely. Based on the results of this study, random assignment of intact units is not an acceptable alternative to random assignment of individuals. In fact, when using intact units, non-random selection techniques were considerably more effective at controlling for potentially confounding influences than randomly assigning existing classrooms to treatment and control groups.

A major objective of NCLB is to hold schools accountable for student achievement including closing the achievement gap and raising standards of student academic proficiency. While the majority of individuals support the underlying values of NCLB, not all agree its accountability policy is reasonable. The most problematic issue is the mandate that schools be held accountable to ensure all students perform at grade level when the methods used by states to determine grade level proficiency preclude the attainment of this standard. The unintended consequences of a policy that does not align theoretically with established educational best practice inevitably result in frustration of educational practitioners and often unwarranted condemnation of the educational system.

Learning can be enhanced through the use of technology; however, learning does not take place simply because technology is used. Neither can technology fully replace the intelligent human guidance of a skilled science teacher. Technology improves science instruction in a laboratory situation when the learning activity is aligned with the learning objectives of the course, is well-structured, and when the equipment and materials to be used are well-managed. In order for this to happen the science teacher must have a clear understanding of the true nature of science, both the content and the process. Both teacher and students need to obtain a working knowledge of the function and utility of the technology. And while technology enhanced instruction does tend to motivate students to participate in learning activities, learning is more likely to occur when the technology being used becomes a transparent tool in the learning process and not the main focus of the activity.

Christensen, S., Davies, R. Hanks, J., Harris, S., & Bowles, B. (Manuscript in Review). Parental Perceptions of the Teaching Profession: Factors that Predict Parental Encouragement of Students to Enter the Teaching Profession

Hanks, J. H., Ferrin, S. E., Davies, R. S., Christensen, S. S., Harris, S. P., & Bowles, W. B. (in-press). Law and policy impacts on teacher attrition in public education: Data suggesting a new focus beyond the silver bullets of targeted STEM and other salary increases.

Analyzing the learning analytics from a course provides insights that can impact instructional design decisions. This study used educational data mining techniques, specifically a longitudinal k-means cluster analysis, to identify the strategies students used when completing the online portion of an online flipped spreadsheet course. An analysis of these results showed that students did tend to follow a specific learning strategy as they completed this course. However, students also self-regulated to some degree, based on the topic and context of specific lessons. These insights not only improve our understanding about the students taking the course, but they also provide guidance for how the instructional design of the course might be improved. Of note is the practical value of this proof-of-concept study in using educational data mining to improve the instructional design of a course.

While there are a plethora of models and approaches for designing instruction (Allen, 2012; Dick & Carey, 1990; Gibbons, 2014; Merrill, 2002), there are only three basic steps in the instructional design process (Gagne, Briggs, & Wager, 1992). First, define expected learning objectives for the instructional unit. Second, determine how you will assess the anticipated learning outcomes. Third, design instructional resources and activities that will help students obtain the knowledge, skills, and abilities the course was intended to facilitate.

This study examines factors that influence STEM university students’ willingness to consider teaching as a career. Although very few of these students initially consider this profession, we identified four factors using predictive modeling that are strongly associated with these students’ willingness to consider teaching and their belief that teaching might be their best career option. Results indicated that STEM university students were more likely to consider teaching when they believed teaching is something they would be good at, others encouraged them to be a teacher, when family encourages them to teach, and when teachers they know inspire them. Results from this study indicate that small salary bonuses would likely not entice students in STEM subjects to become teachers. Less impactful factors included gender and individual beliefs about the respectability of the profession. Additionally, this study found students less likely to consider work conditions for teachers when making career choices.

The purpose of this study was to identify differences in perceptions between three stakeholder groups—principals, K-12 teachers, and parents—regarding the effect of workplace conditions on teacher attrition. All three groups agreed that workplace conditions are important, but they disagreed about (a) which workplace conditions are most problematic for teachers, (b) the magnitude of these problems, and (c) the degree to which these problems may contribute to teachers leaving. The greatest disagreements occurred in perceptions of (a) teachers’ involvement in decision-making, (b) protection of teacher preparation time, (c) administration’s management of student discipline, (d) adequacy of resource availability, (e) the degree to which a trusting and supportive school environment existed within the school, and (f) whether teachers’ expectations were reasonable. Overall, principals believed that work conditions are relatively good for teachers, while many teachers disagreed with these perceptions.

This study examines factors that influence high school students’ willingness to consider teaching as a career. Using predictive modeling, we identified five factors that are highly predictive
of a students’ willingness to consider teaching and their belief that teaching is their best career option. Results indicated that high school students were more likely to consider teaching when they had
confidence in their ability to be good teachers, when family members and others encouraged them to become teachers, and when they felt their community supported teachers. Most of those who considered teaching thought of themselves as average students. Less impactful factors included gender and pay. Additionally, this study found students less likely to consider work conditions for teachers when making career choices.

Young, J., Davies, R., Jenkins J., & Pfleger , I. (2019). Keystroke Dynamics: Establishing Keyprints to Verify Users in Online Courses. Computers in the Schools. 36(1). 1-21.

This paper explores student learning strategies in an introductory spreadsheets course. Student study habits were tracked at a level of detail not available in previous research. Detailed data were collected regarding reading, video watching, actions in practice assignments, references to assignment instructions, and actions in graded assignments. The analysis indicates that student strategies cluster into four primary learning groups. The study provides insight into how instructors can develop their courses and lectures in ways that better match the learning strategies of their students

This study compared student workload and perceived value of coursework assigned for a matching set of semester and term general education courses at [name of University]. Statistically significant differences in workloads were found between most semester and term courses. While term workloads were slightly lighter in general, both could be called “university lite,” in that students did not spend the expected two hours outside of class per hour in class. Math and physics courses came closest to meeting the expected workloads, which tended to remain constant between semesters and terms. Differences in the value students reported for homework varied significantly by the autonomy of the instructor to adapt his own course section. Some of the curricular differences between sessions might be attributed to efficiencies instructors incorporated for shorter sessions without affecting overall course quality. Typically, reading- and writing-intensive courses showed the most negative impact when offered in a term format. The findings from this study suggest that while some subjects lend themselves well to a compressed-time format, not all courses are suited to being taught in this way.