Reverse Engineering Exercises

Reverse Engineering [“RE”] is a process whereby students take an existing research paper and de-construct it back into the ‘Pitching Research’ Framework.

This RE research activity provides clarity and structure for understanding and interpreting academic work. In it’s ‘Reverse Engineering’ form, ‘Pitching Research’ provides a succinct and adaptable framework for all students at all levels of study, to collect, evaluate and communicate the essential elements of an existing research article or new research idea.

RE Pitch - Step by step guide

Research students will need to read and process a lot of scholarly literature and quite often remembering what you’ve read can be hard, if not impossible. A good way to deal with the problem is to create a template for every paper you read.  Imam Salehudin, undertaking his PhD at the University of Queensland Business School (UQBS) has explored using the reverse engineering process to extract key meaning from academic papers for his literature review. Here is a brief abstract from his paper:

The first point in a reverse engineer pitch is the Working Title. Different from the usual template, the working title of this pitch should be the full citation reference of the reverse engineered paper.[…] This way, the reverse engineer pitch can be easily referred later during the actual writing of the literature review.

The second point is the Basic Research Question. Identifying this item is the key information of any academic literature. Our understanding of a literature would be incomplete without a clear identification of what is the research question being asked. Usually, this element can be found in the
abstract or introduction.

The third point is the Key Papers. Identifying key papers is important in exploring other literature relevant to the paper being reviewed. Usually, key papers are mentioned early in the introduction section and repeatedly throughout the article. Key papers should be state of the art, so
most likely they are published less than five years from the date of publication of the paper being reviewed.

The fourth point is the Motivation/Puzzle. This part relates to the background of the research question and the overall purpose of the paper. Some papers cited real world phenomenon as their motivation while others referred to some gap in the theory. It is usually described in the introduction
section and also referred again in the conclusion.

The fifth point is the Idea. This part focuses on the core “intellectual drive” of the paper. In a quantitative paper, this idea will appear in the literature review or the methodology section as research hypotheses. In a more qualitative paper, this idea will appear in the introduction section as the aim and objectives of the said paper.

The sixth point is the Data. This part highlights the data used in the study. As an exception, conceptual papers do not use any data. However, in most cases, research papers will describe in detail the type, size and source of the data in the methodology section.

The seventh point is the Tools. This part focuses on the analytical method used to generate the findings. For quantitative papers, this part will focus on the statistical analysis; while for qualitative papers, this part will describe the analytical framework. It is important to note that citing the software used in the study (i.e. SPSS) does not constitute identifying the analytical tool since
most statistical software can aid in multiple analytical methods. This information is usually described in more detail in the method section.

The eighth point is the What’s New. This part delineates the novelty of the paper being reviewed. The novelty of a paper should be on the idea of the study, but sometimes it is focused on the data or tools used in the research. Most likely, this part is identified clearly in the introduction and conclusion. If not, the reader also must carefully shift for it in the literature review or methodology section. Researchers can also present this novelty in a Mickey Mouse Venn diagram (see figure 1 [link to the original paper below]).

The ninth point is the So What. This part elaborates the impact and implication of the paper for stakeholders. A single study may have different impacts and implications for each identified stakeholders. In some cases, this part can be easily identified from the introduction and conclusion. However, more often, the paper did not identify this part explicitly. Therefore, the reader must construct possible impacts and implications as inferred from relevant information in the introduction, discussion and conclusion.

The tenth point is the Contribution. This part identifies the academic contribution of the paper. It answers specifically what is the research implication of the paper and what further questions that can be asked based on the findings of the study. Similar to the previous part on impact and implication, readers may have to construct their own understanding of the contribution of a paper. Sometimes, the actual contribution of a paper as perceived by the reader is different than what is mention in the introduction or conclusion section of the paper being reviewed.

The eleventh point is the Other Consideration. This final part focuses on any additional reflection on the paper. In the case of a reverse engineer template, this part identifies the key findings of the paper being reviewed. Key findings include support or refutation to existing theories and new insights on the phenomenon being explored. Most likely, readers can find this information
described in the result and discussion section.

For more information, check out  Imam Salehudin’s original paper, where he discusses the process in detail, gives the example and reflects on the whole experience.

``PR Lite``

“PR Lite” procedure will help you find a viable and worthwhile research idea through creating a “mini” pitch. If you are yet undecided on the topic of your next research, try following these 5 steps:

“Step 1: Identify a narrow field of research that you want to explore.

Step 2: Find recent published papers in this field of research on which to execute the RE pitch exercise (e.g. start with 2 papers).

Step 3: Do the RE pitch exercise with each paper.

Step 4: Compare and contrast the papers (based on the RE pitch templates) to find a potential aspect to develop a new research idea, built around a “mini pitch” comprising: (i) Basic research question (Part (B)); (ii) Idea (Part (E)); (iii) Data (Part(F)); and (iv) Tools (Part (G)).

Step 5: Search online aiming to discover whether your new research idea is substantially covered by anyone else’s work. If not, you can use the new idea to develop your own full “pitch”. If yes, repeat Step 3 and Step 4 using the new paper that you discovered aiming to find other new things. You can eventually repeat your work from Step 1, if needed, until you truly find a new research idea.

If you want to know more about the steps, check the following paper:

Nguyen, Bao and Faff, Robert W. and Haq, Mamiza, Pitching Research Lite: A Reverse-Engineering Strategy for Finding a New Research Direction (February 1, 2017). Available at SSRN: https://ssrn.com/abstract=2909549 or http://dx.doi.org/10.2139/ssrn.2909549