Teaching best research practices
to middle and high school students
.png)
The Scientific Method
Lesson Overview and Learning Goals
In this lesson, we invite you to question the relationship between research and critical thinking and reflect on the importance of these activities in and outside of the classroom.
Good science is methodical: it's objectives, scope, design, and expectations are clearly defined by investigators. The scientific method is essential in research as we challenge previously established conclusions and strive to formulate future directions for research. Science is driven by curiosity and healthy skepticism!
After exploring this material, you will be able to:
-
Define and apply scientific terminology used in research design
-
Evaluate research questions in terms of their clarity, specificity, and feasibility
-
Develop your own research questions and testable hypotheses
-
Interpret results and draw conclusions from your own research
Introduction
In the modern world, we are constantly being bombarded with large amounts of information containing contradictory and often rather confusing information. Additionally, with recent advancements in technology, information has now become accessible at the touch through a variety of digital platforms (i.e. social media, published articles, news channels, etc.). As such, it has become increasingly important to learn the fundamentals of critical thinking to properly analyze the validity of all the information presented to us daily!
In understanding the scientific method, you can become a conscious consumer of information and implement aspects of critical thinking into various aspects of your daily life.
The scientific method aims to minimize subjectivity as it outlines the necessary steps that must be followed to produce a valid research result. Moreover, it provides us with the building blocks in which proper experiments and scientific inquiries can be built on. As you will learn, the scientific method is essential in research as we challenge previously established conclusions and strive to formulate future research proposals.
Additionally, the scientific method also plays a critical role in the global standardization of scientific research among the scientific community.
Important Vocabulary
Before we dive into the steps of the scientific method, we must first gain an understanding of various terms that will be discussed. Below is a list of important vocabulary for this section.
Hypothesis:
An educated guess of the outcome of a given study; what one might expect to conclude at the end of the study
Theory:
A system of ideas, based on evidence, used to explain a phenomenon in our natural world.
Variable:
A study feature that is not definite (it can change!)
Independent Variable (IV):
A variable that does not rely on another
-
I.e. Time studied for exam
Dependent Variable (DV):
A variable dependent on the value of an independent variable
-
I.e. Mark on the exam that was studied for
Confounding Variable:
A variable, outside those of which are being studied, that influences the relationship between the dependent and independent variables.
Operational Definition
(of a variable):
Designates meaning to a variable by describing the way in which it will be measured.
Population:
All elements containing data that a researcher wishes to draw conclusions on collectively as a whole.
Sample:
A subset of the population that a researcher uses for data collection.
Important: A sample should not contain bias and should accurately depict the characteristics of a population as a whole.
What is the Scientific Method?
The scientific method is a set of principles and procedures for pursuing knowledge and conducting research. It involves recognition and formulation of a problem, the collection of experimental data through observation, and the testing of hypotheses.
Research, as defined previously, is a process of examining sources to draw conclusions, establish beliefs and facts, and develop new questions to drive future investigations
Knowledge Check!
What is a hypothesis?
Hover your mouse over the magnifying glass to reveal the answer.
An educated guess of the outcome of a given study; what one might expect to conclude at the end of the study
A Model of the Scientific Method
The Research Question
The study purpose often informs and goes hand-in-hand with a research question: the question around which you center your research. Like the study purpose, the research question speaks to a particular problem that has to be resolved.
Here are some characteristics of an effective research question:
-
Clear: Provides specifics so that the audience can grasp the study goals without additional explanation
-
Focused: Defines the scope of the study, identifying variables to be measured, and establishing the limits of the study (this avoids collection of data that is not strictly necessary)
-
Arguable: Upon conclusion of a study based on your posed question, the results should be open to debate rather than accepted facts
A good research question will require synthesis and analysis of ideas and sources prior to reaching an answer. Complexity of research questions means they often cannot be answered with a simple “yes” or “no.” Referring back to the example of tomato plants, to align with the purpose is to identify which fertilizer is optimal for plant growth. With this blueprint, we can devise the following question:
(Question A): “Will brand name fertilizer be more effective than homemade fertilizer?”
This is an example of a simple research question, and following the study, may be answerable through a “yes” or “no.” However, if your objective is well focused, so too should your research question be. For instance, you may instead ask:
(Question B): “How much better is brand name fertilizer than homemade fertilizer (such as eggshells and banana peels)?”
This question cannot be answered like the previous question with a “yes” or “no,” and will require more detailed analysis. Although different, Questions A and B both specify an independent variable: type of fertilizer.
How could you find the answers to the two above questions?
The study purpose often informs and goes hand-in-hand with a research question: the question around which you center your research. Like the study purpose, the research question speaks to a particular problem that has to be resolved.
Here are some characteristics of an effective research question:
-
Clear: Provides specifics so that the audience can grasp the study goals without additional explanation
-
Focused: Defines the scope of the study, identifying variables to be measured, and establishing the limits of the study (this avoids collection of data that is not strictly necessary)
-
Arguable: Upon conclusion of a study based on your posed question, the results should be open to debate rather than accepted facts
A good research question will require synthesis and analysis of ideas and sources prior to reaching an answer. Complexity of research questions means they often cannot be answered with a simple “yes” or “no.” Referring back to the example of tomato plants, to align with the purpose is to identify which fertilizer is optimal for plant growth. With this blueprint, we can devise the following question:
(Question A): “Will brand name fertilizer be more effective than homemade fertilizer?”
This is an example of a simple research question, and following the study, may be answerable through a “yes” or “no.” However, if your objective is well focused, so too should your research question be. For instance, you may instead ask:
(Question B): “How much better is brand name fertilizer than homemade fertilizer (such as eggshells and banana peels)?”
This question cannot be answered like the previous question with a “yes” or “no,” and will require more detailed analysis. Although different, Questions A and B both specify an independent variable: type of fertilizer.
Plant B, grown with brand name fertilizer, is taller than Plant A, grown using homemade fertilizer. In fact, it is 30 cm taller. This observation may be sufficient in answering Question A: “Will brand name fertilizer be more effective than homemade fertilizer?” Based on the observation, it is reasonable to conclude that brand name fertilizer is more effective than fertilizer which is homemade. You may not even need to undertake the calculation of height difference to answer the question.
However, in answering Question B: “How much better is brand name fertilizer than homemade fertilizer,” a calculation is required because simply noting that Plant B is 30 cm taller than Plant A does not answer our question. Instead we can perform a comparative calculation, i.e. determine to what extent is Plant B taller than Plant A. Or similarly, determine the magnitude of difference between the two heights:
90 cm / 60 cm = 1.5
The plant grown using brand name fertilizer grew 1.5 times taller than the plant grown using homemade fertilizer.
This example illustrates how a well defined research question can help guide the development of the protocol and design of study, specifically in selecting effective variables to study.
However, as you likely already know, you will not be able to jump from your research question to data analysis! There are many steps along the scientific method, and following formulation of your research question is formulation of a hypothesis.
Hypothesis
A hypothesis is a prediction of what you expect to happen in regard to a research question that is made without the support of evidence. In short, a hypothesis is an educated guess. Most crucially, your hypothesis must be testable!
Here are three essential criteria of a testable hypothesis:
-
There must be a possibility to prove that the hypothesis is true
-
Similarly, there must be a possibility to prove that the hypothesis is false
-
The results of the hypothesis must be reproducible, i.e. if the same study methodology is conducted is a different place and/or time, the results of that study should match yours
Like the study purpose and research question, a compelling hypothesis should clearly define the topic and the focus of the experiment. Lastly, you should try and write your hypothesis as an “if-then” statement. Here is an example referring back to our tomato plants problem:
Hypothesis: If brand name fertilizer is used as the nutrient supply for tomato plants, then the plants will grow taller and yield more fruit than if homemade fertilizer is used.
This type of “if-then'' statement ensures that your independent variable (type of fertilizer) and dependent variable (operationalized as plant height and/or yield of fruit) are explicit.
Designing a Study to Effectively Test the Hypothesis
Formulation of your study purpose, research question, and hypothesis will have streamlined your study. The next step in the scientific method is to design your study to effectively test your hypothesis. There are two main types of studies to consider between:
Study type one: correlational study
-
Identifies a general relationship between variables
What is a correlation? A correlation is a very important statistical concept, and is frequently mis-represented in scientific research. Correlations describe any statistical relationships between variables. It is very important to note that just because two variables are related does not mean that one causes the other.
Here is an interesting example to illustrate the concept of a correlation:
The red line shows recent trends in chicken consumption in the US.
The black line shows recent trends in US oil imports.
As illustrated in the figure, between the years of 2000 and 2009, these two variables seem to be on a similar trajectory! Are they related? Yes - although the units for both variables are different, they tend to vary in direction and magnitude. In other words, the two lines mostly overlap. These two variables are undoubtedly correlated.
Is there a causal relationship between these two variables? No - changes in chicken consumption in the US did not cause changes in oil imports, or vice versa. This type of relationship is known as a spurious correlation: a relationship completely by coincidence.
To undertake a correlational study, you do not need to define your variables as “independent” or “dependent.” These terms are typically reserved for when your objective is to investigate a causal relationship between variables. Instead, you simply measure your variables to identify whether they are correlated. For example, following measurement of ice cream sales (variable A) and daytime temperature (variable B) during the summertime, you may conclude that ice cream sales and temperature are correlated, i.e. when one increases, so does the other.
Study type two: experimental study
-
identifies causal relationships between variables
In experimental studies, unlike correlational studies, variables are manipulated by investigators in order to determine if (and often how) the chosen independent variable affects the dependent variable.
Let’s take one last look at the example of tomato plants. To answer the previously posed Question A: “Will brand name fertilizer be more effective than homemade fertilizer” You will need to supply one plant with homemade fertilizer and a different plant with brand name fertilizer. If the plant supplied brand name fertilizer grows taller than the one supplied homemade fertilizer, you can conclude that the type of fertilizer (the independent variable) is causally related to plant height (the operationalized dependent variable). This would be true in theory, but in practice you must be much more careful with how you undertake your experiment.
Let’s define a new term: confounder. A confounder (or confounding variable) is a variable that distorts (or hides) the true relationship between your independent and dependent study variables. Confounders introduce bias into study designs because of this “hiding” of real relationships. Moreover, because confounders are neither your independent or dependent variable, they are often referred to as “tertiary” variables.
Let’s take a closer look at this idea of a confounding variable. Perhaps your experimental set up looks like this:
Looks like brand name fertilizer worked better than homemade -or did it? Look closer, you’ll notice that your study design has a clear confounder: water! If you only watered the tomato plant which was supplied with brand name fertilizer, then you cannot determine whether it was actually the fertilizer which caused the plant to grow tall; It could have been the extra water that the plant received. With the presence of water as a confounder in this experiment, the true relationship between fertilizer type and plant height is unknown. The only solution is to ensure that both plants are treated equally with respect to the confounder, this means that either you water both plants or you water neither.
This concept of ensuring confounders are equally distributed between your study groups or participants, is extremely important in research. If confounders were not assessed and controlled for in scientific studies, it would be impossible to undertake clinical trials for drugs or investigate potential negative effects of lifestyle habits, such as smoking, lack of exercise, or a poor diet.
Collecting Data
After developing a study that effectively tests your hypothesis, you must collect data. You must consider the tools you will be using to collect data and also how well these tools measure your variables.
Qualitative vs Quantitative Data
We are interested in identifying how changes to the independent variable influence the dependent variable. There are many ways we can measure this relationship using both qualitative observations and quantitative measurements.
Qualitative observations typically involve using your senses (sight, touch, taste, smell, hearing) to describe the dependent variable and it produces qualitative data. For example, “the flower is red” is qualitative data. Qualitative observations are usually more subjective meaning that different people may make slightly different observations, resulting in different data. In relation to the flower example, another person may say, “the flower is burgundy.” For this reason, qualitative data is typically less reliable than quantitative data.
Quantitative measurements result in data represented by numbers and units. For example, “the bucket weighs 500 grams (g)” is quantitative data. Collecting qualitative data requires specialized tools but is much more objective than qualitative data. For example, anyone who measures the weight of the bucket from the previous example should come to the same answer of 500g (or a very similar answer). As such, it is preferred to take quantitative measurements whenever possible.
Types of Qualitative Measurement Tools
-
Physical measurements
-
Uses tools such as thermometers, scales, or meter sticks to measure changes in a dependant variable
-
More useful in studies not using human subjects/participants
-
-
Questionnaires
-
A series of questions where subjects typically select one or multiple predefined answers (i.e. true/false or multiple choice format)
-
-
Interviews
-
An interviewer has a face-to-face conversation with the subject and asks questions relevant to the hypothesis
-
Have the potential to generate a lot of data, but since the subject can respond open-endedly, answers between subjects may vary.
-
-
Simulations
-
Create a simulation that mimics a real life event or phenomenon
-
Rate the subjects performance during the simulation
-
Conclusion
Data collection can be a long and expensive endeavor. The average cost of enrolling a single patient in a clinical trial is $41,117 and many trials will enroll hundreds or even thousands of patients. As such, it is essential to ensure that you are selecting data collection tools that will allow you to effectively test your hypothesis
Analyze Data
Raw data is the term used to describe recently collected data that has not yet been processed. Oftentimes raw data cannot easily be understood simply by looking at it and therefore must be organized in a way that is more digestible. The process cleaning and transforming data so that it can be more easily understood is called data analysis.
Imagine you just started driving and wanted to know when the roads are least busy so that you could easily practice. You hypothesize that the roads are least busy at lunch while most people are at work. You decide to test your hypothesis by sitting on your front lawn and counting the amount of cars that drive by in the morning, at noon, and in the evening. You are especially scared of driving near transport trucks so you also decide to keep track of when they drive past. Below is the raw data you collect at the end of the day.
From these tallies, the best time to drive is not entirely clear. If we count up the tallys, it becomes a bit easier.
We could also add together the two types of vehicles for each of the three times of the day to help make our decision.
Finally, we may want to graph to aid in our interpretation.
Hopefully you can begin to understand the importance of data processing and analysis. From this graph, we can see that the road is busiest in the evening but has the most transport trucks at noon. This information can help inform when you decide to practice driving. Also, you may be intrigued by these results and develop a new question that you wish to study: why are there more transport trucks on the road at noon? This will require a new hypothesis and more data collection!
Data Analysis in Real Life
Oftentimes data collection is much more complex than what we saw in the example above. Data may be collected by multiple different people all over the world at many different times, making it even more important to effectively process and analyze data. Typically this involves summarizing data using graphs or tables as we did above.
Interpret Results and Draw a Conclusion
The final step of the scientific method is to interpret your data in order to draw a conclusion regarding your initial question. Often your conclusion will also lead to new questions making the scientific process more of a scientific cycle.
Once your data is analyzed you will want to ask yourself the following questions:
-
Do the results confirm my hypothesis?
-
Do the results weakly or strongly confirm my hypothesis?
-
Do the results actually make sense? Would a friend or co-worker agree?
-
How does this new knowledge fit into the pre-existing body of knowledge?
-
What is the impact of this data? Why is it helpful?
-
What new questions can be developed from this data?
Drawing a conclusion typically involves answering some or all of these questions in written oral format.
Written formats:
-
Research journals
-
Books
-
Websites
-
Magazines
-
News
-
Posters
Oral Formats:
-
Presentations a scientific conference
-
Radio
Lesson Conclusion
As should now be clear, the scientific method is a set of steps that help answer a question. The scientific method can also be thought of as a cycle, since the answer to one question often leads to further inquiry.
While scientists employ the scientific method to identify new diseases, develop therapeutics, and study the human body, it is also used frequently in everyday life, often without even realizing! As such, we encourage you to familiarize yourself with specific steps of the method so that you can consciously and critically analyze novel information.
Check out the video below for more information on the scientific method:
References
George Mason University. (2020). How to Write a Research Question. Retrieved from: https://writingcenter.gmu.edu/guides/how-to-write-a-research-question
The Open University. (2017). 13.4.2 Research objectives. Retrieved from: https://www.open.edu/openlearncreate/mod/oucontent/view.php?id=231§ion=8.6.2
Tyler Vigen (n.d.). Spurious Correlations. Retrieved from: https://tylervigen.com/spurious-correlations
Moore et al. Estimated Costs of Pivotal Trials for Novel Therapeutic Agents Approved by the US Food and Drug Administration, 2015-2016. JAMA Intern Med