Data Analysis for Astronomy and Physics

Full course, open to BSc and MSc students. Course page, QIS/LSF Link

Course Description

The topics covered in the lecture are listed here. The corresponding lecture notes will be available before the lecture. Please work through the materials before the lecture! The focus of the lecture will be on the application of selected content. The lecture material will be deepened based on pre-calculated examples. Furthermore, the concrete applications should stimulate discussion and the methods should be questioned.

It is important that you come prepared to the lecture so that we have sufficient time to deepen the teaching material.

Content presented during the lecture will be recorded (video, audio, and text) and made available electronically afterwards.

The script is written in English and is provided as PDF and as a Mathematica Notebook. However, the respective German technical terms will also be mentioned during the lecture.

You can also view Mathematica Notebooks without a Mathematica license using the CDF Player.

Syllabus

Syllabus (PDF), Mathematica Notebook

Still unsure if the course is right for you? The following quiz (PDF, NB) offers an initial self-assessment. If you can answer the questions, you are already familiar with some concepts covered in the lecture. If you are uncertain about many answers, the lecture will help you deepen your understanding.

The videos are also collected in this YouTube Playlist.

For in-depth preparation, I try to link external material on selected topics. Examples include educational videos, such as those by Prof. Jörn Loviscach, and others.

Lecture Schedule

1. Introduction to Data & Data Visualization

Date: April 22, 2025 (Q & A: April 29, 2025)

Lecture Notes:

• Organization: Syllabus (PDF), Mathematica Notebook
• Unsure whether to take the course? Take the Quiz
• Introduction to Data (PDF), Mathematica Notebook
• Data Visualization (PDF), Mathematica Notebook

Video Lectures:

• Introduction to Data (Part 1) (24:04)
• Introduction to Data (Part 2) (13:17)
• Data Visualization (Part 1) (21:24)
• Data Visualization (Part 2) (23:57)
• Data Visualization (Part 3) (09:46)
• Data Visualization (Part 4) (10:04)

Additional Materials:

• NEW: TED Talk - How to spot a misleading graph (4:09)
• CSE512 Data Visualization (Winter 2014), University of Washington
• Khan Academy:
• Creating and interpreting scatterplots
• Box and whisker plots
• Frequency tables & dot plots
• Creating a histogram
• Interpreting a histogram

Assignments: Assignments are shared and submitted via GitHub Classroom.

• Invitation link for a starter assignment to learn GitHub basics
• Invitation link for Assignment 1: Intro to data structure and data visualization

Data:

• Data set (CSV)
• Explanation (RTF)

Q & A Materials:

• Plot-data: 2.9,2.45,2.5,3.5,3.7,3.0,1.75,0.75,0.3,0.2,1.9,1.6,0.5,-0.5,-2.8
• Monthly temperature records (CSV)
• Temperature station list (CSV)
• Labor cost chart (PNG)

2. Statistics

Date: April 29, 2025 (Q & A: May 6, 2025)

Lecture Notes:

• Statistics (PDF)
• Statistics (NB)

Video Lectures:

• Statistics (Part 1: Statistics of location) (34:47)
• Statistics (Part 2: Statistics of scatter) (15:47)
• Statistics (Part 3: Samples & bias) (21:07)

Additional Materials:

• Variance, standard deviation (13:37)
• Calculating variance, standard deviation (13:13)
• Normal distribution, central limit theorem (14:58)
• Probability concepts, frequency, Bayes, Laplace
• Kolmogorov axioms of probability
• Variance, standard deviation of a random variable (36:55)
• Averaging reduces variance and standard deviation (21:30)
• Three coins; expected value of the standard deviation of the sample (18:46)
• Standard deviation of lifetime (14:14)
• Expected value, variance, standard deviation for discrete and continuous distributions (18:35)
• Expected value, variance, standard deviation of three coins (13:13)

Q & A Materials:

• Monthly temperature records (CSV)
• Temperature station list (CSV)
• Labor cost chart (PNG)

3. Probabilities

Date: May 6, 2025 (Q & A: May 13, 2025)

Lecture Notes:

• Probabilities Part 1 (PDF)
• Probabilities Part 1 (NB)
• Probabilities Part 2 (PDF)
• Probabilities Part 2 (NB)

Video Lectures:

• Probabilities A 1/5
• Probabilities A 2/5
• Probabilities A 3/5
• Probabilities A 4/5
• Probabilities A 5/5
• Probabilities B 1/4
• Probabilities B 2/4
• Probabilities B 3/4
• Probabilities B 4/4

Additional Materials:

• Probability concepts, frequency, Bayes, Laplace
• Kolmogorov axioms of probability
• Khan Academy:
  • Summarizing center of distributions
  • Summarizing spread of distributions
  • Sampling distributions
  • Intro to theoretical probability
  • Simple probability examples
  • Experimental probability
  • Permutation formula
  • Combination formula
• Bozeman Science: Standard Error
• Additional probability examples and explanations

4. Probability Distributions

Date: May 13, 2025 (Q & A: May 20, 2025)

Lecture Notes:

• Probability Distributions (PDF)
• Probability Distributions (NB)

Video Lectures:

• Probability Distributions (Part 1) (16:11)
• Probability Distributions (Part 2) (43:10)
• Probability Distributions (Part 3) (24:53)
• Probability Distributions (Part 4) (13:42)

Additional Materials:

• Normal distribution, central limit theorem, sketch of a derivation (67:20)
• Normal distribution in OpenOffice.org, probability density, cumulative distribution function
• Khan Academy:
  • Shapes of distributions
  • Comparing distributions
  • Normal distributions
  • Binomial probability
  • Poisson distribution
  • Variance of differences of random variables
  • Central limit theorem

Assignment 2:

• Invitation link for DAAP assignment 2: Astronomical Data

5. Hypothesis Testing

Date: May 20, 2025 (Q & A: May 27, 2025)

Lecture Notes:

• Hypothesis Testing (PDF)
• Hypothesis Testing (NB)
• HTs for categorical variables (PDF)
• HTs for categorical variables (NB)

Video Lectures:

• Hypothesis Tests (Part 1) (21:33)
• Hypothesis Tests (Part 2) (24:35)
• Hypothesis Tests (Part 3) (17:38)
• Hypothesis Tests (Part 4) (09:46)
• Hypothesis Tests for Categorical Variables (Part 1) (09:27)
• Hypothesis Tests for Categorical Variables (Part 2) (18:35)
• Hypothesis Tests for Categorical Variables (Part 3) (11:08)
• Hypothesis Tests for Categorical Variables (Part 4) (18:43)

Additional Materials:

• Khan Academy:
  • Confidence intervals (one sample)
  • Significance tests (one sample)
  • Significance tests and confidence intervals (two samples)
  • Analysis of variance (ANOVA)
  • ANOVA 1: Calculating SST
  • ANOVA 2: Calculating SSW and SSB
  • ANOVA 3: Hypothesis test with F-statistic
  • Margin of error tutorials
  • Standard error of the mean
  • Hypothesis testing and p-values
  • One-tailed and two-tailed tests
  • Z-statistics vs. T-statistics
  • T-statistic confidence interval
  • Inference for categorical data (chi-square tests)

6. Correlation

Date: May 27, 2025 (Q & A: June 3, 2025)

Lecture Notes:

• Correlation (PDF)
• Correlation (NB)

Video Lectures:

• Correlation (Part 1) (17:15)
• Correlation (Part 2) (35:46)
• Correlation (Part 3) (23:23)
• Correlation (Part 4) (07:52)

Additional Materials:

• Correlation and causality

Assignment 3:

• Invitation link for DAAP assignment 3: Bayesian statistics, samples statistics and tests

Q & A Materials:

• CII_01 (FITS)
• plane.dat (CSV)
• figure_0_noise_2.h5 (HDF5)
• figure_0_noise_5.h5 (HDF5)
• images.csv (CSV)
• imagelabel.txt
• radio-map.fits - artificial 5 by 5 grid of spectra
• IC342.12motf.chan.fits - spectral CO (1-0) map of IC 342