- Entry self-test.
Only statistical results will be shown. - This is not a `statistics' course, but about probability
and uncertainty focused on inferential issues.
(Not a collection of formulae, or of tests “with russian names”
)
- What is “Statistics”?
→ Lecture at CERN (Lecture 1, sl. 5-8).
- C'è statistica e statistica
- “An unsophisticated forecaster uses statistics as a drunken man uses lamp-posts − for support rather than for illumination” (Andrew Lang, cited in Famous Forecasting Quotes)
- La statistica secondo Trilussa
- “Claims of discoveries based on sigmas”
- HASCO
Summer School, sl. 1-20.
(Vedi anche Incontri di Frascati 2012, incluso video e supporto R)
- HASCO
Summer School, sl. 1-20.
- A recent,
- Continuation of the entry test: → on reading errors,
and uncertainties on digital readings.
- An Android app to check your personal(!) skill in reading
analog instruments (in steady conditions):
ErroriLettura.apk
No dogmatism, please! - Other apps from Google app store:
- Vernier Caliper Simulator (try to read the tenths without using the vernier)
- Length Fraction Calculator, another useful app to check the ability to read the tenths (and less).
- An Android app to check your personal(!) skill in reading
analog instruments (in steady conditions):
ErroriLettura.apk
- P-values and ... supercazzole
→ See exchange of mails in the pdf file posted on the web page (not linked). - First intro to the R language
- Some examples from R
- Producing Simple Graphs with R
- RStudio
- About falsification and its "statistical variations": p-values.
- Some issues of the lecture in R
- A foundamental question (if you do not try to answer
yourself you will not
understand the issues behind it!)
- rnorm(1, sample(1:2)[1], 0.5)
Question: from which of the two 'μ' does the resulting number come from?
- n=100000; hist(c(rnorm(n,1,0.5), rnorm(n,2,0.5)), nc=100)
- variant just to remark that statisticians have
strange ideas about
what “prob” is!
n=100000; hist(c(rnorm(n,1,0.5), rnorm(n,2,0.5)), prob=TRUE, nc=100)
- rnorm(1, sample(1:2)[1], 0.5)
- esempi_IdF2012.R
- gaussiane_IdF2012.R
- chi2_IdF2012.R
- A foundamental question (if you do not try to answer
yourself you will not
understand the issues behind it!)
- R at Coursera (running courses, or starting soon, or still available):
- Data Science Specialization [10 short courses(*)]:
- Mathematical Biostatistics Boot Camp 1 (perhaps still available)
- Statistics One
- https://www.coursera.org/specializations/genomic-data-science
- What can you do with R? Quite a lot.(*) Here is the the alphabetic
list of the
MANY packages available
(*)But most R packages are written in C! So those who wrote R or contribute to it didn't do because they were/are unable to program in C, but because doing normal work in C is heavy and very inefficient.
- P-values
- What p-values do not mean
- Doing mistakes (and cheating!) with p-values:
- “A lot of what is published is incorrect” (Lancet editorial)
- p-hacking, or cheating on a p-value (and how R can help...)
- Significant (at xkcd - well done!)
- P > 0.05? I can make any p-value statistically significant with adaptive FDR procedures
- The Ease of Cheating With Statistics
- False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant.
- “Instead of heeding impressive-sounding statistics, we should ask what scientists actually believe”
- Psychology Journal Bans Significance Testing
- Guide to the Expression of Uncertainty in Measurement (GUM),
by ISO: see here;
(Browsable version at iso.org: the 'decalogo' is in section 3.3.2) - About errors in reading analog instruments (and related topics):
- R code to show the effect of rounding
- residui_da_lettura.R:
- residui_da_lettura.eps,
- residui_da_lettura.png (converted with 'convert', although direct png from R is preferable)
- ErroriLettura.apk (reproposed)
- GdA, Errori e incertezze di misura - Rassegna critica
e proposte per l'insegnamento
(vedi qui),
pp. 7-23 e 82-84.
- Android app to check your ability to interpolate between marks: ErroriLettura.apk (screenshot).
- J. D. Mollon, A. J. Perkins, Errors of judgement at Greenwich in 1796, Nature 380, 101 - 102 (14 Mar 1996) (for the pdf seehere, or local copy)
- Ole Rømer and his determination of the speed of light (interessante sito in italiano)
- Dava Sobel, Longitude.
- R code to show the effect of rounding
- About Frequentistic
“Confidence Intervals” and
“Confidence Levels” that
they do not imply levels of confidence
(in the human sense!):
- G. Cowan, "Statistical data analysis", Chapter 9, on "Statistical errors, confidence intervals and limits" ("/dott-prob_22/CI/" -- not linked: go to the Ciclo 22 and guess how to reach the web page): for self study (we shall come back to it later).
- “Definizione dell'intervallo di confidenza” (da dispense in italiano, autori omessi...).
- Do check on your preferred books and lecture notes.
- About the meaning of upper/lower limits (e.g. the famous pre-LHC MH > 114 GeV at 90% C.L.
- Why the (tens of) thousand confidence limits of the last 3-4 decades to not agree with expectations from coverage?
- Neyman about the confidence intervals (“Carry out your experiment... ”)
(Fisher used to call the confidence intervals “that technological and commercial apparatus”) - The ultimate confidence intervals calculator (Android version: Ultimate_CI_Calculator.apk).
- GdA, “Bayesian Reasoning in Data Analysis”, Chapter 1
(Based on first chapter of CERN Yellow Report) - GdA, “From Observations to Hypotheses: Probabilistic Reasoning Versus Falsificationism and its Statistical Variations”, physics/0412148v2.
- Wikipedia, P-Value [expecially 'Frequent misunderstandings' and the article "Historical background on the widespread confusion of the p-value: a major educational failure" di R. Hubbard e J.S. Armstrong, therein cited (local copy)].
- R. Hubbard and M.J. Bayarri,
Confusion over measures of evidence (p's) versus errors (α's)
in classical statistical testing,
American Statistician, 57 (2003) 171-178
(available here;
local copy)
Additional matter on the subject;:- P-values are not error probabilities (Hubbard and Bayarri)
- Why We Don’t Really Know What “Statistical Significance” Means: A Major Educational Failure (Hubbard and Scott Armstrong)
- Why P Values Are Not a Useful Measure of Evidence in Statistical Significance Testing (Bayarri and Lindsay).
- Further Wikipedia references
- Farneticante (per diversi aspetti)
lettera aperta al direttore del CERN, apparsa
il 15 gennaio su Il Fatto Quotidiano:
a parte i diversi punti discutibili, sui quali non entro,
- Esercizio: trovare i due errori nella frase “Per semplificare, dire che il bosone di Higgs è stato osservato con intervallo di confidenza 6 sigma, vuol dire che c’è una probabilità di 4 miliardesimi che il bosone di Higgs non esista.”
- Soluzione.
- Arbitrary probabilistic inversions implicit in the construction
of frequentistic 'confidence intervals', that do not
provide a "confidence".
[Very insightful quote from
Mathematical Biostatistics Boot Camp 1 (Lecture 8 video, starting
at about the 20th minute):
“ If you happen to be taking a statistics
test, there's a trick to get around this mental games associated with fictitious
repetitions of experiments and so on. The trick is just to say we're 95%
confident the interval contains mu, and statisticians have said
well that's enough hedging to count as a legitimate instance
of the strict definition. So, if you're taking
a statistics test don't say there's a 95% chance that the interval
that I just calculated contains mu, cuz your teacher might
yell at you about that. But if you say you're 95% confident,
they'll begrudgingly give you credit.” --
see here
for the complete, official transcript of the video.]
- The dog and the hunter.
- Implicit assunsions and case in which they do not hold.
- Objective methods are often arbitrary and unjustified (and often the solveproblems difference from those practitioners have).
- Pure empirical information is not Science!
Concerning the misterious sqrt(2), try to play with radice_di_due.R. - Remarks of standard (old?) analysis methods in first years
laboratory:
- rule of the “half scale spacing”;
- “theory” of maximum bounds (and their propagation);
- “always draw error bars in plots!”;
- lines of minimum and maximum slope;
- on the propagation of the “statistical”
errors:
- contradiction with “standard definition of probability” (that is apply probabilistic formulae to object about which probability statements are esplicitly forbitten);
- issues of evaluating the sigmas (it is not necessary to make many measurements!);
- oversimplifications due to not taking into account correlations;
- and, anyway, linearization is sometimes a bad assumption.
- Learning from data, learning about models and their parameters, forecasting future observations: the inferential predictive process.
- Deep source of uncertainty: causal links. The essential problem of the experimental method.
- From true values to observations... and back.
- A simple experiment (with most issues of real experiments): the six box problem.
- Comparison of probability of black/white from box known composition wrt box of unkown composition: Ellsberg paradox (name not mentioned during the lecture).
- What is probability?
- Probability and bets.
- GdA, Errori e incertezze di misura - Rassegna critica e proposte per l'insegnamento
- GdA, Introducing Bayesian Reasoning in Measurements with a Toy Experiment
- GdA, 2005 CERN Academic Training (second lecture);
- GdA, Bayesian reasoning (or CERN Yellow Report): Chapter 1.
- xkcd comics on betting:
- Did the sun just explode?;
- (Opera's) Neutrinos
- And another one to remind that correlation does not necessarly imply causation:
- Some training with R
- Remarks on descriptive statistics, and reminders on the different meanings of "statistica" (particularly in Italian, see C'è statistica e statistica)
- On the “n-1” in the definition of the
standard deviation calculated from a sample
- interference between descriptive and inferential statistics;
- irrelevant correction is n is large;
- when n is such small to make a great difference (e.g. 2 or 3) is more important our prior idea on what σ could possibly be that what we learn from data.
- xkcd comics with a boxplot showing an outlier
- Uncertainty and probability. Degree of belief or “a quantitative measure of the strength of our conjecture or anticipation”
- About the importance of state of information in evaluating
probabilities (probability is always
conditional probability!):
- two variants of the 3 box problem (one of which corresponds to the famous Monty Hall, whose 'explanation' is made exagerately long in the wiki);
- the two envelopes problems:
- equiprobability is often misused!
- do not confuse wishes and beliefs!
- Meaning and role of subjective probability (that is far from being arbitrary). Anticipation of possible objections.
- Basic rules of probability, and meaning of the relation between joint probability, conditional probability and probability of the conditionand (certainly not a 'definition' of conditional probability!).
- When frequentistic gurus talk about the “probability” of a true value to be in an interval: see here.
- GdA, 2005 CERN Academic Training: last part of second lecture; third lecture till p. 25.
[→ play special attention to probabilistic statements about the meaning of the excess]
- Reports from CMS and Atlas;
[note in the abstract “This excess is not statistically conclusive...”: when is an excess statistically conclusive?] - Some theoretical speculations
- Again on models transferring the past to the future
(motivated by a conversation with a colleague about “smoothing
background”):
- Cern Academic Training, Lecture 2, pp. 13-20
(see in particulare the Hume's quote); - A joke from “Plato and a platypus walk into a bar... − Understanding philosophy through jokes” (recensione su Scienza per Tutti): The scientist, his wife and the shorn sheep (equivalent to that about the philosopher. the physicist, the matematician... and the cow).
- Cern Academic Training, Lecture 2, pp. 13-20
- Probability and odds.
- Basic rules of probability from coherence.
- Expected gain.
- Remarks on combinatorics.
- Set, events and rules of probability.
- On the interpretation of P(E) = Σi P(E|Hi)×P(Hi): probability of probability.
- Conditional event and fourth basic rule of probability derived from coherence (and more ramarks on its meaning).
- Events dependent/independent in probability.
- Back to the six box problem and comparison of the box
of unknow composition Vs that of precesely known composition.
- More on Ellsberg's paradox, and on Ellsberg.
- Probability of the sequences of colors (WW, WB, BW, BB)
from the two boxes (after reintroduction):
- the probabilities are all equal for the box of known composition;
- Instead, for the box on unknown composition
they aren't:
- during the extractions we are learning something;
- → the subsequent colors are non independent!
- P(W1,W2) = P(W1)×P(W2|W1)
- Uncertain numbers and... “random numbers”
- Remarks about randomness (à la von Mises).
- Probability functions of discrete uncertain numbers: f(x) and F(x)
- Introduction to Monte Carlo “random numbers” generators:
- reweighing of events;
- 'hitting' the steps of F(x) with (pseudo-)randomly generated numbers;
- extension to the continuum;
- hit/miss technique for continuous variables for
- random number generation;
- evaluation of integrals.
- Monte Carlo sampling in R:
- Some examples of Jags used (via rjags) just as a MC generator
- R. Scozzafava, Incertezza e probabilità (Zanichelli), 1.1-1.16.
- GdA, Probabilità e incertezze di misura: Chapter 3, 4 and 6.
- GdA, Role and meaning of subjective probability: some comments on common misconceptions, MaxEnt2000, physics/0010064.
- Stanford Encyclopedia of Philosophy: Interpretation of probability.
- D. Lewis, A Subjectivist's Guide to Objective Chance (local copy), Published in Richard C. Jeffrey (ed.), Studies in Inductive Logic and Probability, Vol. II. Berkeley: University of Berkeley Press, 263-293. Reprinted with Postscripts in David Lewis (1986), Philosophical Papers. Vol. II. Oxford: Oxford University Press, 83-132.
- A very nice page on uncertainty and probability: Marguerite Yourcenar, Memoirs of Hadrian → Look Inside!: p. 5.
- Playing with a circle:
- Bertrand "paradox", with a practical test:
- “Draw a chord at random”;
- list the instructions (pseudocode) to extract chords “at random” starting from a uniform random generator.
- Estimating π by sampling.
- A curious game (trowing stones), as a very first introducion to MCMC.
- Bertrand "paradox", with a practical test:
- Logical and 'stochastic' (probabilistic) dependence/indipendence.
- Bernoulli trials: geometric and binomial (and Pascal).
- Poisson process: relation between Poisson distribution and esponential distribution.
- Probability function of the sum of the numbers resulting from two dice,
with comments on R
(distr_due_dadi.R, including command to save the plot as eps file) - More on exponential distribution and its relation with exponential and geometric. Decay life times.
- General scheme of the ditributions deriving from the bernoulli process: schema_distribuzioni.pdf
- How much to believe that X in within E[X]+-σ? General considerations (and caveat!); Markov and Cebicev disequalities.
- Properties of the 'operators' E[] and Var[] under linear transformations.
- Bernoulli theorem and its misinterpretations and mistifications.
- Generalities on probability distributions of continuous variables.
- Uniform and triangular distribution (both symmetric and asymmetric, stressing their conceptual and pratical importance!)
- Remarks on the propagation of uncertainties, expecially in the case of asymmetric distributions. 'Eye opener' example: somma_triangolari_asimmetriche.pdf (see arXiv:physics/0403086 for details.)
- Normal distribution.
- Propagation on uncertainties: genaral consideration and general multivariate formulae for discete and continuos variables.
- Examples of application to the sum of uniform continuous variables and of asymmetric triangular distributions. (See remarks and reference three items above.)
- Propagation by sampling (`Monte Carlo').
- Sum of two triangular-asymmetrically distributed variables
(see above) performd in R (using triang.R):
- somma_triangolari_asimmetriche_sampling.R
- somma_triangolari_asimmetriche_sampling.png (example of resulting histograms)
- Sum of two triangular-asymmetrically distributed variables
(see above) performd in R (using triang.R):
For references etc. see previous course and dispense in italiano.
- Details on Exponential and decays.
- Gaussian tricks to evaluate expected values and variance of pdf "assumed approximately normal" (to be used cum grano salis).
- On the general formula to 'propagate' the pdf
one input quantity to the pdf of one output quantity:
- proposed exercises on simple transformations, line linear, x^2, sqrt(x), x^4. Graphical interpretation of the 'distorsion' of the densities;
- a curious transformation, Y=F(X), to show that a quantity defined as the cumulative of another one is unformely distributed between 0 and 1.
- Linear transformations of independent variables:
general rules and Central Limit Theorem (CLT).
Remark: it is assumed to be applied to cumulative distributions (defines on the real axis also for discrete distributions). - Some applications:
- Expected value and variance of the binomial, of the Pascal and of the Erlang (a Gamma with k integer).
- Reproductive property of the Binomial and of the Poisson distribution, and its relevance for the use of the CLT.
- Distribution of the arithmetic average;
- A simple rough Gaussian random generator;
- Measuremt errors.
- Words of caution: we are dealing with probabilities not with certainties! (If you don't like to live in a continuous state of uncertainty take in consideration to mobe to the mathematics department...)
- Remark on the use of probability theory to perform propagations of quantities: we have to assume that probability statement can be applied to the (uncertain) values of physics quantities. (Intellectual schizophrenia not tolerated!)
- From Bernoulli process to random walk... and more:
- From binomial distribution to expected random walk;
- 'Gambler ruin' problem (just sketched).
- Pallinometro (Quinconce de Galton, or Bean machine, also available as Android app): comments on the 'didactic paradox' (of the ol, mechanical boards, like the Roman 'pallinometro'.)
- Brownian motion in 1D.
- Measurement errors (errors, not uncertainties!) as a random walk the signal space.
- A (limited) random walk in the velocity space: euristic derivation of the Maxwell distribution on velocities.
- A Maxwell distribution in 2D: Rayleigh distribution.
- Use of the Rayleigh distribution, with s=1, to make a 100% effcient renerator of (pairs of) Gaussian numbers from pairs of uniformly distributed numbers.
For references etc. see
previous course
and dispense in italiano.
Intereresting links on Bayes, Laplace (“the man who did everything”), Turing and more (as a preparation to the inferential part of the course):
- "The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code(*), Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy":
- Sharon Bertsch McGrayne: "The Theory That Would Not Die" | Talk at Google
Software recommended (not as important as Jags/rjags for applications in Physics):
- On previous lectures:
- Clarification about the distribution of r (distance from the origin) in the case of points in a plain whose coordinates are i.i.d. ~ N(0,1): r_distr_unif_Vs_norm.R.
- Exercise to get the pdf of tn, time to wait n events in a Poisson process of constant intensity r. (Hint: use the general tranformation rule with the Dirac delta.)
- The “Laplace's” Bayes rule.
- Application to the six box toy model:
- analysis of the experiment with R;
- analysis with Hugin Expert:
For references etc. see
- Second part of the slides of talks at Incontri di Fisica and HASCO (see above);
- GdA, Teaching statistics in the physics curriculum: Unifying and clarifying role of subjective probability
How to install VGAM and to use rrayleigh()
[Unfortunatly it does not work with install.packages("VGAM"),
at least in the version I have. The following has worked under Linux]
- Download VGAM_1.0-0.tar.gz (or later version) from the developer site or from the CRAN.
- Enter in R as root, then
install.packages('VGAM_1.0-0.tar.gz', repos = NULL, type="source")
[check that VGAM_1.0-0.tar.gz is in the working directory, or use the proper path] - q()
- Then, as an example, as normal user:
- library(VGAM)
- n= 10000; r <- rrayleigh(n, 1); phi=runif(n, 0, 2*pi)
- x <- r*cos(phi); y <- r*sin(phi)
- plot(x,y,pch=19, cex=0.2, col='cyan')
Trovato il mitico articolo del Corriera sul protone che aveva la venerabile età di 1025 anni:
- More on the six box problems:
- Simulating extractions. Comparisong between the probabilities of the next extraction calculated using probability theory with those extimated by past frequences.
- Analysis of the extractions without reintroduction.
- “Untangled boxes.”
- Inferring white ball proportions.
- The Bayes' billard
- A-B-C of parametric inference (with flat priors)
- Inferring p of Bernoulli processes
from: the sequence of successes/failures;
the number of successes.
- Laplace' succession rule;
- probabilistic meaning of the expected value of f(p);
- on the evaluation of the probability from past relative frequency
- Inferring λ of a Poisson distribution from the number of counts.
- Inferring μ of a Gaussian distribution
from a single observation:
- f(xf|xp) and the (not really) 'famous' sqrt(2) factor in the standard deviation.
- Inferring p of Bernoulli processes
from: the sequence of successes/failures;
the number of successes.
References
- GdA, Bayesian Reasoning
- GdA, 1999 CERN Yellow Report
- GdA, Dispense
- Inferring "Bernoulli'p" from different kinds of experiments. Meaning of E(p). Special cases. Predictive distribution. Beta distribution as prior conjugate.
- Inferring λ of the Poisson distribution. Special cases.
- Gamma distribution as conjugate prior of the Poisson distribution.
- Relation of the Gamma to other distributions.
- Multivariate distributions: overview and definitions.
- Bivariate normal distribution.
- Multivariate normal distribution.
- GdA, Bayesian Reasoning
- GdA, Probabilità e incertezze di misura (dispense)
| The p-value 'revolution'(*) |
| The American Statistical Association's statement |
|
(*)As explained in the
Nature paper,
“This is the first time that the 177-year-old ASA has made explicit recommendations on such a foundational matter in statistics.” |
- Covariance matrix of linear combinations.
- Exercise on A4 paper: Exercise_A4_1.pdf.
- Repeate exercise using matrix formalism.
- Warnings concerning propagation of uncertainties.
- On the combination of "confidence intervals" (or of "probable intervals").
- Linearization.
- Other exercise on A4 paper: Exercise_A4_2.pdf.
- Special case of monomial espressions: propagation of relative uncertainties (with special case of independent variables).
- Inferring μ of a Gaussian from a single observation (assuming σ):
- case of flat prior;
- Gaussian prior conjugate: combining result;
- predictive distribution
- The Gauss derivation of the Gaussian.
- Introduction to the treatmenet of uncertaintyes due to systematic.
- Special case of uncertainty on `the zero' of an instrument
('offset uncertainty'):
- global uncertainty;
- correlation induced on the result of measurements obtained with the same instrument.
References
- GdA, Bayesian Reasoning
- GdA, 1999 CERN Yellow Report
- GdA, Dispense
- For some relevant pages (in Latin) of the Gauss' derivation of the Gaussian see here.
- GdA, On the use of the covariance matrix to fit correlated data
- GdA, W. de Boer and G. Grindhammer, Determination of αs and the Z0 Mass From Measurements of the Total Hadronic Cross-section in e+e− Annihilation
- GdA, Asymmetric Uncertainties: Sources, Treatment and Potential Dangers
- GdA and M. Raso, Uncertainties due to imperfect knowledge of systematic effects: general considerations and approximate formulae
- References etc.
- Dispense, parte 4.
- C. Andrieu at al. An introduction to MCMC for Machine Learning, pdf.
- As far as OpenBUGS is concerned see
info here, plus
the following examples
- Model: gauss_Nm_sigma_pred.txt
- Dati: gauss_Nm.dat.txt
- Inits: gauss_Nm.in.txt
- Resulting chain ('coda'): bugs_out.txt and bugs_ind.txt
- Example of how to analyze the chain in R: gauss_Nm_sigma_pred.R
- Example of how to generate data with R: genera_dati.R
- References etc.
- GdA, Bayesian Reasoning
- GdA,Errori e incertezze di misura - Rassegna critica e proposte per l'insegnamento
- GdA, On the use of the covariance matrix to fit correlated data
- C. Andrieu at al. An introduction to MCMC for Machine Learning: pdf.
- GdA, A Multidimensional unfolding method based on Bayes' theorem
- GdA, Improved iterative Bayesian unfolding
- R code
- importance-sampling.R
- JAGS and
rjags: Try to install them
- Getting Started with JAGS, rjags, and Bayesian Modelling (link not checked, but it looks ok)
(vedi referenze della volta scorsa)
- More on fits: arXiv:physics/0511182
- Examples of rjags:
- simpleMC_1.R
- simpleMC_2.R
- simpleMC_3.R
- inf_lambda_pred.R (model: inf_lambda_pred.bug)
- inf_lambda_pred_N.R (model: inf_lambda_pred_N.bug)
- inf_r.R (model: inf_r.bug)
- inf_r_bkg_measured.R (model: inf_r_bkg_measured.bug)
- Other scripts (or files of R commands):