Introduction

Draft Version 556 (Wed Nov 9 17:41:04 2005)

Computers are as important to scientists as telescopes and test tubes
- Analyze problems that are too complex for traditional means
- Simulate things that can't be studied in laboratories
Many scientists now spend much of their professional lives writing and maintaining software
- A quarter of graduate students in science and engineering spend 25-50% of their time programming
But most scientists have never been taught how to do this efficiently
- It's a long way from the loops and arrays of first year to simulating bone development in foetal marsupials…
- Like being shown how to differentiate polynomials, then expected to invent the rest of calculus
This course will teach you how to design, build, maintain, and share programs more efficiently
- Focus: tools and techniques appropriate for half a dozen people working together for a year
  - Everything you do at that scale will also make you more productive when you're working on your own for a week
- Will not turn you into a computer scientist
  - Far too many of them around anyway
- Instead, goal is to teach you the equivalent of good laboratory technique for computational science
  - The 20% of ideas that account for 80% of real world use
  - Software carpentry, rather than software engineering
Comment on this slide

Meeting Standards

Experimental results are only publishable if they are believed to be correct and reproducible
- Equipment calibrated, samples uncontaminated, relevant steps recorded
- In practice, almost always rely on the professionalism of the people doing the work
How well do computational scientists meet these standards?
- Correctness of code rarely questioned
  - We all know programs are buggy, but when was the last time you saw a paper rejected because of concerns over the quality of the software used to produce the results?
- Reproducibility often nonexistent
  - How many people can reproduce, much less trace, each computational result in their thesis?
The times they are a-changing
- Standards for computational work can only go up
- Change can happen almost overnight
  - Like the American car market when German and Japanese imports appeared in the 1970s
Comment on this slide

Improving quality improves productivity
I.e., the tools and techniques you must adopt to produce better code also help you write more code, faster
Comment on this slide

User stories
- Important part of designing user interfaces for mass-distribution software
- Helps make discussion of features and usability more concrete
Harry
- 27; B.Sc. in zoology
- Did an introductory Fortran course nine years ago, and attended a workshop on web-based bioinformatics tools when he started his job
- Now developing fuzzy pattern-matching algorithms for Genes'R'Us, a biotech firm with labs in four countries
Ron
- 24; B.Eng in mechanical engineering, now doing an M.Sc. part time
- Did C in first year; has been using MATLAB ever since
- Modeling thermal degradation (a.k.a. “melting”) of firefighters' helmets
Hermione
- 34; Ph.D. in physics
- Took two courses on C and two on numerical analysis as an undergrad, and a computer graphics course as a graduate student
- Now in charge of the 5-person flywheel braking group at Yoyodyne Inc.
Ginny
- 22; finished a B.Eng. in chemical engineering last year, now doing an M.Sc. in biochemistry
- Did C in first year, and has built a personal web site (static HTML only)
- Thesis topic is improving the yield of organic fullerene production
Albus
- 47; Ph.D. in mathematics; studies large random graphs using both analytic and computational techniques
- Undergraduate degree in Mathematics and Computer Science in the 1970s; programs mainly in C++
- Professor at Euphoric State University; former chair of graduate studies
Comment on this slide

Adapted from [Spolsky 2004] and [McConnell 1997]
- 1 for “yes”, 0 for “no”, and -1 if you don't understand the question
So:
- Do you use version control?
- Can you rebuild everything in one step?
- Do you have an automated test suite?
- Do you build the software, and run the test suite, daily?
- Do you have a bug database?
- Do you use a symbolic debugger?
- Do you use a style checker to ensure that your software is written in a uniform, readable way?
- Can you trace everything you release back to the software that produced it?
- Do you document as you program, and keep your documentation in your source files?
- Can you set up a development environment (including any libraries you need) on a fresh machine without heroic effort?
- Do you have a schedule with small binary milestones?
- Do you estimate how long tasks will take before you start, and compare that with how long they actually took?
- If you're working with others, or developing software that other people may use:
  - Do you test your interfaces using paper prototypes before implementing them?
  - Is there a searchable archive of discussions about the project?
  - Do team members write and share small tools for automating common tasks?
  - Does your schedule allow for infrastructure development, training, sick time, etc.?
And your score is?
Comment on this slide

So why are we where we are?
- It's difficult to learn these things from academic computer scientists
  - CS research is more concerned with rapid prototyping than with reliability
- People are naturally sceptical of innovation
  - Particularly after they've seen a few bandwagons roll through
  - Glass's Law [Glass 2002] : any new way of doing things initially slows you down
- You only have to be as good as the competition
  - American auto makers in the 1970s
This course's approach:
- Introduce some basic tools
  - Students immediately see benefit of taking the course
  - Tools can be used to manage the course itself
- Show students how to build tools like these
  - Where “how” includes both what goes into the software, and how to create it
  - Solidifies understanding of tools' capabilities and limitations
  - Makes discussion of technique more concrete
- Show students what else they can do with their new skills
  - The right way to tackle issues that come up over and over again
Key point: avoid overload
- People who already know these things tend to underestimate how hard they are to learn
- No point preaching to the converted
- Try instead to move the middle of the bell curve to the right
Why is the course called software “carpentry”?
- Because it focuses on the details of the craft
- If software engineering is about building an electronic version of the Channel Tunnel, this stuff is the equivalent of putting an extension on the house
Comment on this slide

Source code.

Interactive commands.

Program output.

Program errors.