Image

HONR 259C "Fearfully Great Lizards": Topics in Dinosaur Research

Fall Semester 2017
Scientific Research Examined & the Anatomy of a Scientific Paper


Interns Ben Otoo and Nicole Ridgwell studying and photographing the Late Jurassic dinosaur fossils at Dinosaur National Monument

Key Points:
•"Science" is more than a body of knowledge. It is a way of approaching questions about the natural world.
•Observations (data) about nature are used to develop hypotheses: questions you can test
•Science progresses by the subjecting of hypotheses to tests (experiments) to see if you can reject them; those you cannot reject are provisionally accepted until new information is available.
•Scientific studies are presented as papers: publications where scientists state their observations, methods, analytical results, and conclusions. Other researchers can build upon this information; sometimes supporting it, sometimes rejecting it.


The Hypothetico-Deductive Method
Science is not simply "a body of knowledge". Rather, it is the systematic acquisition and application of knowledge about the structure and behavior of the physical universe gained via empirical evidence through observation, measurement, and experimentation. It can be described as a type of inquiry into nature characterized by the availability of empirically testable hypotheses.

Science starts from several important observations about the world around us:

Science is thus a self-correcting mechanism: it contains within its operation the means to get rid of old, less-accurate models and replace them with more accurate ones.

Argument: a connected series of statements intended to establish a proposition, consisting of one or more premises which support the conclusion. (In the case of Science, the premises should be a series of hypotheses tested against observations.) Many, many fields fields of intellectual endeavor (politics, philosophy, marketing, etc.) relay on arguments. However, Science is distinguished (in a subset of fields: we can add some others like history, economics, etc. in here) by relying on not merely argument butindependent evidence: in other words, it is empirical.

There is much talk about the "Scientific Method", often characterized as a hierarchy of observations --> hypotheses --> theories --> laws. This is in some ways an oversimplification, and in others misses the point. These are more the products of Science, not their method. Perhaps a better descriptor of the "Scientific Method" is simply: Physical Evidence and Reasoned Logic (PEARL).

That physical evidence goes by several names: observations, data, measurements, all meaning the same thing: qualitative or (more commonly) quantitative attribute of a phenomenon. In principle, different observes should be able to make the same measurements/descriptions and find the same value.

Of course, we face the issue of uncertainty: there are limitations of observations (for example, the accuracy and resolution of instruments) so we will not always find the same exact value. Additionally, there are probabilistic aspects to the nature of the Universe, especially at the quantum level.

Raw observations are good, but we need to do more with them. In other words, there must be some form of data analysis. The observations are compared to each other in some fashion (normally some form of mathematical or statistical plot) in order to discover potential patterns, and from that to test different hypotheses.

Collectively, raw observations and analytical results are what we call evidence. We use this evidence to infer what is happening; that is, evidence is used to get to conclusions.

After observing some phenomenon, a pattern often emerges. We can state this pattern formally as an hypothesis. Thus, contrary to its colloquial use, an hypothesis is NOT an "educated guess", but rather "a formal statement of a pattern that appears to exist in a set of observations." In contrast to theories, hypotheses are primarily about the pattern itself, not about an explanation of the pattern.

Unfortunately, humans cannot help but see patterns: castles in the clouds, faces in random objects, "lucky streaks" in games, etc. Not all perceived patterns are real! So to evaluate whether a pattern that we perceive (an hypothesis) might be true, we must test it. We refer to this as "submitting it to falsification": subjecting the hypothesis to some evaluation where it could in principle be shown to be incorrect. Not all hypotheses are falsifiable (synonym: testable): some are purely subjective (e.g., "chocolate is better than vanilla") or involve metaphysical qualities or entities which cannot be measured (e.g., "true justice is superior to true wisdom"). We might hold these as important concepts, but they are outside Science. Indeed, to paraphrase the late philosopher Christopher Hitchens, "What can be asserted without evidence can be dismissed without evidence".

Other hypotheses can be potentially falsifiable with total knowledge of the universe, but we are currently [and perhaps forever] incapable of evaluating (e.g., "the flesh of eurypterids ('water scorpions', extinct for 252 million years) is an effective cure for athletes foot"). Ideas of this second sort are speculations: nothing wrong with them as such, but they aren't particularly useful.

Even good scientific hypotheses remain mere speculations until submitted to a test (often called an experiment). An experiment (i.e., a test of falsification) must be constructed in such a way that the hypothesis could yield observations that demonstrate that the hypothesis is false. For example, we can speculate that "my horse can outrun any horse in the world." However, until we gather evidence to test it, we won't know if this is actually the case (no matter how much we want to believe it.) A rather simple experiment for this: a horse race. If another horse outraces it, then the hypothesis is falsified.

This procedure is called the "hypothetico-deductive method", and is basically what experiments in Science are all about. To put it in its basic form "If you were wrong, how would you know it?"

Note that part of developing a good experimental design is actually phrasing your hypothesis properly. For example, we might be interested in the presence of fossils of ceratopsids (horned dinosaurs, like Triceratops) in rocks the last 15 million years of the Cretaceous Period in continental Africa. If we state our hypothesis "there were latest Cretaceous African ceratopsids", than how many observations do we need to make to demonstrate this is true? Does a single observation with negative results show us there is no ceratopsids in latest Cretaceous Africa? No. How about ten negative results? A thousand? Until we have excavated all latest Cretaceous rocks from Africa, we cannot demonstrate that there are no ceratopsids. However, by stating the hypothesis as "there were no ceratopsids in latest Cretaceous Africa", then a single positive observation of a ceratopsian is all we need to confidently overturn (falsify) this hypothesis.

The above is an example of employing a null hypothesis. The null represents a form of the hypothesis that must be rejected before we even accept a phenomenon exists. If we cannot reject the null hypothesis, there is no reason to think that the phenomenon in question is worth considering. If, instead, we find we can reject the null (in the instance above, finding a latest Cretaceous ceratopsid in Africa), than we can go on and examine the situation in more detail.

The hypothetico-deductive method shows that we can confidently reject hypotheses, but that we cannot "prove" them in an absolute philosophical sense. As the number of observations and experiments which fail to reject an hypothesis increases, we can be more and more confident in its truth. However, ideas in Science are only provisionally accepted: that is, we often use statistical measures of confidence (plus/minus readings, error bars, and other demonstrators of degree of support). Uncertainty is a staple part of Science. However, some ideas are so overwhelmingly well-supported that to reject them at present is perverse: these are what we call "facts". (Similarly - using the example above - if we made many excavations in latest Cretaceous continental African rocks and continuously failed to uncover ceratopsid fossils but find plenty of other dinosaurs, we will provisionally reject the idea of African horned dinosaurs, but recognize that a single fossil could overturn this rejection.)

The following (from Thomas Kida's Don't Believe Everything You Think) are a useful set of characteristics for thinking like a scientist:

Historical Sciences
Something worth noting concerning the issue of paleontology, historical geology, evolutionary biology, astronomy, and other sciences which are concerned primarily with events which have already happened: i.e., historical sciences. These are concerned with actual particular events and cases just as much as general patterns (as opposed to, say, particle physics or organic chemistry, where all collisions of the same particles or folding of the same protiens are identical). As a consequence, although we can perform some kinds of experiments or observations under controlled circumstances to duplicate the past conditions, in general we are looking at evidence of the past event.

That doesn't make it any less scientific, nor does it dismiss the ability to do repeatable observations. "Repeatability" in this case is where different observers--or the same observer in multiple different examinations--can make the same observations of the same set of data. (For example, if a scientist asserts that a particular layer of clay shows an enriched abundance of the metal iridium, consistent with an asteroid impact, other scientists can sample the same layer and look for this material, and the original research can resample the material to test that it is there.)

There is a cliché that paleontologists are a kind of "detective", and that isn't a bad comparison. Much like forensics experts in crime scene investigations or medical examiners, we examine a series of data to develop a hypothesis (or multiple hypotheses) to explain the observations at hand, and test these hypotheses against both the currently-known data and additional relevant data we look for.

In general, it is best to follow Carl Sagan's "Baloney Detection Kit":


"It's Only A Theory"
Theory: like "hypothesis", the popular meanings of this ("a guess" or "just made up") is NOT the academic meaning! In academics, a theory is a model or set of rules with broad explanatory power about some particular phenomenon. For example, consider music theory, color theory, game theory, political theory, or economic theory. We don't say "color is just a theory", or "you are just guessing about the existence of music"!! Instead, under this use a "theory" is a"a system of ideas intended to explain something" or a "set of underlying rules" or "operational logic" or (most simply of all) "an explanation".

However, just because something is a theory doesn't mean it is an accurate map of how the Universe really operates. A scientific theory is more specifically a comprehensive framework for describing, explaining, and making falsifiable predictions about related sets of phenomena based on rigorous observation, experimentation, and logic. Scientific theories are not necessarily correct (that is, accurate maps of the Universe), but before we accept them they should at least match our current observations.

A key element of both theories and hypotheses is that they are predictive: that is, by using them you can make estimates before hand about observations not yet done. In the case of historical sciences like paleontology and geology, "predictions" don't necessarily mean "what things will be like in the future", but rather "what will we be likely to discover about a past event, which we have not yet examined".

Contrary to some statements (even by reputable scientific organizations), scientific theories are not limited to describing only large scale ("universal") phenomena nor ongoing phenomena. Thus, "one time" only events (such as the impact theory of the extinction at the end of the Cretaceous or the Big Bang at the beginning of spacetime) or phenomena limited in scope (such as the theory of thermohaline circulation as the major driver of modern climate, or again of the impact cause of the Cretaceous extinction) are indeed subject to theory, and these theories can be explored and tested via parsimony, fecundity, consilience, auxilary hypotheses, and so forth. Nevertheless, the most important (and most fecund) theories ARE about large-scale ongoing phenomena: atomic theory of matter; periodic theory of the elements; special relativity; the germ theory of disease; the theory of evolution by means of natural selection; plate tectonic theory of geology; and so on.

It is very common to find that in certain spheres of Science there are multiple different (and sometimes mutually exclusive) theories proposed for the same observations. Indeed, this is what a lot of scientific research is about: the creation of and testing of new theories and their auxiliary hypotheses. Hence there are fields like theoretical physics in which new models of the operation of the universe and its various components are proposed and assembled based on previous observations, logic, conjecture, and speculation. Other scientists (experimental physicists, observational astronomers, etc.) themselves look for observations that could in principle reject some or all of the components of these theories. This research involves creating sets of experiments whose results will be different under different alternative theories about the phenomenon involved. We can test and reject theories in part in the same fashion as we do hypotheses: that is, by parsimony, consilience, and by tests of falsification.

One last comment: what are scientific laws? Popular accounts of the scientific method suggest a hierarchy of observation -> hypothesis -> theory -> law, but this is not correct. The phrase "scientific law" in the Sciences was largely been abandoned in the 20th Century. Many of the traditional "scientific laws" were simply scientific theories that can be rendered as mathematical equations. As a consequence they tend to deal with relatively simple and more easily measurable phenomena. "Scientific laws" were thus no better nor worse than other scientific theories at withstanding rejection: for example, Bode's law of planetary orbital distance wound up being a coincidence more than a law; Newton's laws of motions only apply to certain gravitational conditions and speeds; and so forth. "Scientific laws" can be useful in some circumstances (e.g., calculating gas pressure, volume, temperature, or number of particles given the other variables using the ideal gas law PV = nRT), but there are many realms of Science where the phenomena are too complicated to be reduced at present to law-like forms. This holds true of much of geology, climatology, biology, and especially behavioral sciences. So be very careful if you hear from someone who proposes some "law" of biology or paleontology or anthropology or psychology!

There is a subset of theories that have withstood substantial repeated tests and modifications and survived if not unscathed, at least strongly supported and the victor against all challengers so far. These incude (but aren't limited to) the atomic theory of matter; the theory of evolution by means of Natural Selection; plate tectonics theory; the germ theory of disease; etc. We honestly don't have a good term to distinguish these theories from the more run-of-the-mill, still-in-play types. (One might suggest facts, and there is some merit there, but a "fact" might be a better synonym for a well-substantiated observation rather than an entire theory.) The British evolutionary biologist Richard Dawkins proposed (in 2009) using the new word theorum for this class of theories. Time will tell if this nomenclature will catch on.

Regardless of what we call them, we can describe this collection ideas as theories which are to the best of our knowledge "true", but this requires the caveat that absolute "Truth" is empirically impossible to find (although it would be extraordinarily perverse to reject these things as being "true" in a general sense without some extraordinary new evidence to the contrary.) These winning theories can be thought of as the all-time prize-winning champions in their respective fields, having faced and defeated all challengers. In principle (indeed, "in theory"! :-) they could be defeated by a newcomer, but the weight of evidence so far is with the champions.


Scientific Knowledge
Here is one of the most important aspects of Science, and one that many people do not appreciate: sometimes, "We do not know" is the best answer. This is often true when we lack potential observations to choose between alternative hypotheses and theories.

On the other hand, many scientific questions (especially technical and precisely phrased ones concerning matters which we can have access to the appropriate observation) have correct answers: that is, they are not matters of opinion (as are questions in a great many of other fields).

Not all viewpoints are equally valid in Science:

Concepts within science are subject to change with new discoveries. That is in fact what scientists DO for a living: make new discoveries!!

Ironically, many people charge Science and/or scientists with acting "like they know all the answers", when the entire discipline is based on the fact that we don't! Alternatively, and also ironically, many people are upset when Science doesn't have "all the answers"

Often, people ask "what do scientists believe." This is the wrong question! Science is not about belief; it is about discoveries and about the methods by which those discoveries were made and tested.

Through Science, we have discovered many aspects of nature. Here are some of the largest level aspects (finer details would be those covered by different content disciplines):


Some Relevant Videos
Below are a seriesVERY short videos by TechNYou, called "This Thing Called Science""
"Part 1: Call me skeptical" (2:02):

"Part 2: Testing, testing 1-2-3" (2:30):

"Part 3: Blinded by Science" (2:45):

"Part 4: Confidently Uncertain" (3:01):

"Part 5: Do the right thing" (2:38):

"Part 6: Citizen Science" (3:34):


The Process of Science: Publications
Scientists are generally not "lone wolves", but instead are members of societies. These may have extremely broad scope (e.g., the Royal Society (an old British group dedicated to all the sciences); the American Association for the Advancement of Science; etc.) which deal with all aspects of Science; or they might have a very specific specialization (e.g., the Pander Society, dedicated to the study of the conodonts, an extinct group of jawless vertebrates known almost exclusively from their tooth-like mouth spikes). Member of these societies gather, discuss and debate ideas, challenge each other, present their discoveries, and publish. These publications are the main way that scientific ideas are presented to the world and preserved for posterity.

Why write & publish papers?

There are at present many different venues in which scientists can present their findings. Here is a sampling of the major ones.

Scientific papers might be very short (1-2 pages), or monographs (dozens to hundreds of pages long), but most are typically between 4 and a few dozen pages long. Monographs are primarily used to thoroughly document a single particular topic: a complete description of the anatomy and biology of a particular species or group of species; a review of the geography or the geology of a particular region; the results of a particular space probe; etc.: in other words, topics with a large amount of observations in a very narrow topic. Very short papers tend to simply announce a new discovery, document an important new observation, or respond to a particular criticism of previous work. The middle range papers are where most the hypothesis testing goes on.

It was once very common to have single-authored papers, or maybe just two authors. These have become fairly rare, and it is common to find papers with a half-dozen or more authors. In fact, in some cases there can be dozens of contributing authors!

How are scientific papers assembled and published? In general, they follow the pattern here:


Anatomy of a Scientific Paper
The actual anatomy of a scientific paper often (although not always) follows a standard model. Unlike creative writing pieces, technical scientific writing is generally not intended to be particularly innovative in style, format, or sequence. Instead, a standardized formula is typically used, to make it easier for the readers to get the relevant information for the paper. (That said, some authors may have a more effective style than others...). Very short papers may not have all the following components, but in general your run-of-the-mill scientific publication will include:

A Special Part of the Materials Section in Paleontology Papers: Systematic Paleontology: The focus of many paleontology papers is, not surprising, the fossils themselves. If the paper focuses on the description of new fossil material--or reevaluation of previously discovered specimens--there is often a special section of the "Materials" called Systematic Paleontology. This section describes the particular fossil(s) that are the focus of the study. In the Systematic Paleontology, the following items are listed:

• The position of the fossil in the hierarchy of the Tree of Life, listed in order from most inclusive to most exclusive
• The name (or names) under which the specimen has been referred to in any previous work (if this is the first time that name is used, it is indicated as such by the statements sp. nov. (for "species nova", new species) or gen. et sp. nov. (for "genus et species nova", new genus and species))
• The specimen number (more about these in the next box) for the holotype and referred specimens
• The horizon (geologic level) and locality (place it was found)
• The etymology of any new names coined
• And the diagnosis (list of distinctive features of the species).

Specimen Numbers & Museum Abbreviations: Each fossil specimen accessioned in a museum has a particular code--a specimen number. This allows researchers to go back to the same specimen to attempt to replicate the observations of previous workers. Each specimen number consists of a museum abbreviation (representing the institution and collection in which the specimen is maintained) and a unique number associated with that specimen (different institutions have different algorithms for this; some simply list specimens sequentially in the order they were accessioned; others with codes that combine the year of discovery and other information.)

There is no way (or reason) for you to memorize all the various museum abbreviations. In fact, any paper which discusses particular specimens is obliged to have a table or paragraph listing the institutional abbreviations used in that particular paper. Here is a crowd-sourcing attempt to create an online list for vertebrate paleontological collections. But just to give some examples of ones you are likely to commonly encounter in this course:

American Institutions: AMNH: American Museum of Natural History, New York, NY; CM, Carnegie Museum of Natural History, Pittsburgh, PA; CMNH, Cleveland Museum of Natural History, Cleveland, OH; DINO (formerly DNM), Dinosaur National Monument, UT; FMNH, Field Museum of Natural History, Chicago, IL; LACM, Natural History Museum of Los Angeles County, Los Angeles, CA; MCZ, Museum of Comparative Zoology, Harvard University, Cambridge, MA; MNA, Museum of Northern Arizona, Flagstaff, AZ; MOR, Museum of the Rockies, Montana State University, Bozeman, MT; SDSM, Museum of Geology, South Dakota School of Mines and Technology, Rapid City, SD; TMM, Texas Memorial Museum, University of Texas, Austin, TX; UCMP, University of California Museum of Paleontology, Berkeley, CA; USNM, National Museum of Natural History, Smithsonian Institution (formerly the United States National Museum, hence the acronym), Washington, DC; YPM, Yale Peabody Museum of Natural History, Yale University, New Haven, CT

Canadian Institutions: CMN (formerly NMC), Canadian Museum of Nature, Ottawa, ON; ROM, Royal Ontario Museum, Toronto, ON; RTMP (sometimes TMP), Royal Tyrrell Museum of Palaeontology, Drumheller, AB

A Smattering of Other Institutions from Around the Globe: GI PST Institute of Geology, Section of Paleontology and Stratigraphy, Mongolian Academy of Sciences, Ulaanbaatar, Mongolia; IVPP, Institute for Vertebrate Paleontology and Paleoanthropology, Beijing, China; MACN, Museo Argentino de Ciencias Naturales 'Bernardino Rivadavia', Buenos Aires, Argentina; MfN (formerly HMN), Museum für Naturkunde Berlin (formerly the Humboldt Museum für Naturkunde), Berlin, Germany; MNHN, Muséum National d'Histoire Naturelle, Paris, France; MUCPv, Museo de Geologia y Paleontologia de la Universidad Nacional del Comahue, Neuquén, Argentina; NHM (also NHMUK, formerly BMNH), Natural History Museum (formerly the British Museum of Natural History), London, UK; PIN, Paleontological Institute, Russian Academy of Sciences, Moscow, Russia; SAM, Iziko South African Museum, Capetown, South Africa; ZPAL, Institute of Paleobiology, Polish Academy of Sciences, Warsaw, Poland

The above description reviews the basic components of a scientific technical paper. There are other sorts of publications in the scientific world which some might confuse for a research report. It is important to bear in mind that these other items are NOT technical research papers, and as such do not have the same weight in terms of their importance in the world of science. In general, these other forms of publication are not tests of hypotheses or reports of new observations. Here are some examples, listed roughly in decreasing order of "significance" in terms of their standing as scientific papers:

Throughout the rest of the program--and indeed, the rest of your academic career--keep these thoughts in mind when evaluating whether a paper you might encounter might be a real technical research report, some other type of scientific account, or something else entirely.

A final note: is peer-review perfect? Of course not, and no one claims that it is! However, the existence of at least this level of checks-and-balance means that scientific technical research reports have had to face a level of criticism that nearly all other forms of publication (news articles, press releases, political tracts, etc.) never have to do before they see the light of day. This is the first main step in the self-correcting nature of Science.

The next main step? If you believe that a previous research article is in error (because of difference in interpretation of methods or observations, or new data that you have uncovered), then you are free to write up a new paper incorporating your new thoughts, and submit it to the same scrutiny. And thus the process continues: what is technically referred to as "reciprocal illumination."


How to Read a Scientific Paper

The above gives you a sense of how scientific research (and other type) papers are constructed. But how should you, as a user of said papers, go about reading them?

One recent tend in undergraduate scientific literacy education is the "C.R.E.A.T.E" approach. This has to do not so much with reading the paper per se, but with attitudes to take when approaching reading science. This (admittedly forced, as are most such things) acronym stands for:

The stereotyped approach to reading a paper is "abstract, then conclusion, then the stuff in between if you are interested in it". This obviously isn't the way the papers are organized, and is a bit unfair to the researchers' work, but at least it is very quick. However, this is by no means the only way. For instance, here is an essay and here a number of perspectives about different approaches to reading a paper. (And here is a more cautionary tale...)

As with most written documents, there is no one single proper way to read them. Find what works best for you.


Some Relevent Videos
"The Scientific Method Made Easy" (9:54, by YouTuber potholer54):

"Introduction to Scientific Journal Literature" (put together by the library of Dalhousie University in Canada):


To Next Lecture.
To Previous Lecture.
To Lecture Notes.

Last modified: 31 August 2017

Image
The systematic paleontology section of the initial description of the Late Jurassic heterodontosaurid Fruitadens (Butler et al. (2009) doi: 10.1098/rspb.2009.1494)