Debugging

Table of Contents

Debugging Process

(1) Describe the problem

  • Write down the problem in a text file, bug tracker or on a piece of paper.
  • Describe the expected result and the observed result.
  • State all relevant facts which are necessary to enable someone else to reproduce the problem, e.g. the operating system and user interaction.
  • If possible, there should be a minimal script with which the problem can be demonstrated.
  • Collect all relevant log files and error messages.

(2) Create a concept diagram

Creating a concept diagram helps to get a grasp on all the moving parts involved in the problem.

(3) Trying stuff until it works

Debugging_ScientificMethod.jpg (Image source: Andreas Zeller, Scientific Method, Creative Commons 1.0)

  1. Analze the systems diagnostic data and its source code.
  2. Formulate a hypothesis ("an educated guess") about the infection chain.
  3. Based on this hypothesis, conduct an experiment to try out a possible step towards the solution.
  4. Based on the observed result, decide whether the hypothesis needs to be refined or rejected.
  5. Repeat these steps until the problem is resolved.

(4) Implement the solution

  • Fix the root cause of the problem, not just the symptoms.
  • If a workaround cannot is necessary, take it as a temporary solution and create a follow-up task.
  • Take preventive measures (e.g. regression tests), so that this class of issue cannot resurface.
  • Refactor the solution along with a clean code checklist.

Tipps and tricks

Scrutinize the error messages

  • End user messages
  • Stack traces
  • Log files
  • Try to write helper scripts for finding the relevant info, e.g. with Bash or Perl.

Search the internet

  • Type in the key parts of the error message the search function of the support forum or a search engine.
  • There are also other search engines than Google.
  • Stackoverflow alone contains a lot of answers.

Keep a debugging logbook

Everything gets written down, formally, so that you know al all times where you are, where you've been, where you're going, and where you want to get. In scientific work and electronics technology this is necessary because otherwise the problems get so complex you get lost in them and confused and forget what you know and what you don't know and have to give up. – Zen and the Art of Motorsycle Maintenance, Robert M. Pirsig

labyrinth-1013625_960_720.jpg

Figure 1: Don't get lost in the debugging maze

For subtle bugs in large programs, the amount of state you need to keep track of can rapidly get out of hand. (…) This is the point at which you should be writing down every single command you type in any relevant prompts, and every single code change (or, since we have technology, obsessively saving the output of `history`, making commits to test branches, and recording the correlation between them). – Nelson Elhage

Divide and conquer

  • Create a new sandbox project.
  • Extract the problematic part from the main project and reproduce the problem there.
  • Remove all code which is not related the actual problem step by step.
  • Then solve the problem.
  • Integrate the solution in the main program.

Understand how the system works

  • What components are involved in the problem and how do they interact?
  • Create a model to reduce the cognitive load (e.g. UML).
  • Reference to the architecture documentation (e.g. arc42).
  • Then use this understanding to locate the problem origin.
  • Read the documentation and manuals.

linux_observability_tools.png

Figure 2: Example for System Analysis utils, by Brendan Gregg

Gather program state information

Debugging_DeducingValueOrigins.jpg (Image source: Andreas Zeller, How failures come to be , Creative Commons 1.0)

Diff Debugging

Ask for help

  • If you get stuck, ask your collegues for help.
  • They might be happy to help you out.
  • If you cannot find a solution together, consider posting the question on Stackoverflow.

http://www.catb.org/esr/faqs/smart-questions.html#before

Take a break

  • go for a walk
  • fresh air
  • drink a coffee
  • relax
  • have a good nights sleep

Print out parts of the source code

  • Write comments with a pen.
  • Mark susspicious sections and sections you don't understand.
  • Draw lines to visualize relations between parts of the system.

Break the Grodian knot

  • You might break the "Grodian knot" by rewriting (parts of) the code.

Debugging stories

References

Effective Troubleshooting

Google - Site Reliability Engineering : https://landing.google.com/sre/book/chapters/effective-troubleshooting.html

Why Programs Fail

TRAFFIC algorithm

(T)rack: The first step of the debugging algorithm is to track the problem report in the bug tracker database. (R)eproduce: The problem needs to be reproducible to the developers in order to get fixed. (A)utomate: The steps to reproduce should be optimized into a minimal test case which can be executed automatically. (F)ind: Now that we can reproduce the problem fastly, we need to find the origins of the defect by tracking the origins of values. (F)ocus: In the search of the problem origins we should focus on the usual suspects, e.g. code smells, known infections and earlier defects. (I)solate: The next step is to isolate the root cause of the infection with the help of the scientific method. (C)orrection: Finally we can fix the defect and re-run all the tests for verification.

GitHub • Impressum