应用多元统计笔记之统计距离

（最近在看图书馆借的《实用多元统计分析》（Ａ约翰逊，迪安　Ｗ威克恩），觉得其用线性代数的方式解释统计的方式很受启发，再加上是借的书，所以希望作些笔记，算是总结以下自己读书的收获，准备用专题的方式把书中的一些概念记下来，同时参考一些其他资料。这也利于以后查阅）

统计距离（statistical distance）是多元统计的基础概念。它是为了解释变异差异和引入相关概念而发展起来的。试想我们测量两个变量，得到一系列数值对，每个变量的数值均值为0。两个变量的测量相互独立。而我们假定变量X1的变异量比变量X2要大。那么在以二维坐标图上（X1为x 轴，X2为y轴）表现为一个形状类似扁长椭圆的散点图。因为x轴的变异量大于y轴（因为X1变异量大于X2），所以在统计上认为将两个坐标赋予相同的权重并不合适。因此我们引进了一个标准化后的坐标 \inline$x_{1}^{*}=x_{1}/\sqrt{s_{11}}$ 和 \inline$x_{2}^{*}=x_{2}/\sqrt{s_{22}}$ 从而定义了统计距离为：
\inline$d(O,P)=\sqrt{(x_{1}^{*})^{2}+(x_{1}^{*})^{2}}$ $=\sqrt{\frac{x_{1}^{2}}{s_{11}}+\frac{x_{2}^{2}}{s_{22}}}$
(引自理查德　Ａ约翰逊，迪安　Ｗ威克恩（２00８）,实用多元统计分析（影印版），清华大学出版社，以下同上）

因此统计距离是对欧式距离的推广（二维是 \inline$d(O,P)=\sqrt{(x_{1})^{2}+(x_{2})^{2}}$ ），当变量间变异量一致时，我们用欧式距离来计算距离，当变异量不一致时我们考虑统计距离。
当统计距离为常量c时， 有以下方程
\frac{x_{1}^{2}}{s_{11}}+\frac{x_{2}^{2}}{s_{22}}=c^{2}$

该方程是一个中心在原点的椭圆方程，与x轴的交点为 \inline$-c\sqrt{s_{11}}$和\inline$c\sqrt{s_{11}}$与y轴的交点为\inline$-c\sqrt{s_{22}}$和 \inline$c\sqrt{s_{22}}$

如果对统计距离在进行推广对任何一个点P(X1,X2)到固定点Y(Y1,Y2)的统计距离是：
\inline$d(O,P)=\sqrt{\frac{(x_{1}-y_{1})^{2}}{s_{11}}+\frac{(x_{2}-y_{2})^{2}}{s_{22}}}$
统计距离还可以推广到啊更高的维度

以上的统计距离公式考虑的是变量之间相互独立的情形，因此有必要考虑变量之间相关的情形，所以引进了旋转角度的方法，如果把相关变量的散点图的坐标轴旋转至符合独立变量的情形那么可以得出
\inline$d(O,P)=\sqrt{\frac{(x_{1}^{'})^{2}}{s_{11}^{'}}+\frac{(x_{2}^{'})^{2}}{s_{22}^{1}}}$
x1’和x2’分别为 \inline$x_{1}^{'}=x_{1}cos(\theta)+x_{2}sin(\theta)$ 和 \inline$x_{2}^{'}=-x_{1}sin(\theta)+x_{2}cos(\theta)$

最终我们得出：
\inline$d(O,P)=\sqrt{a_{11}x_{1}^{2}+2a_{12}x_{1}x_{2}+a_{22}x_{2}^{2}}$

其中a11,a12,a22是由角度θ决定的

同样对于P(X1,X2)到固定点Y(Y1,Y2)统计距离为
\inline$d(O,P)=\sqrt{a_{11}(x_{1}-y_{2})^{2}+2a_{12}(x_{1}-y_{1})(x_{2}-y_{2})+a_{22}(x_{2}-y_{2})^{2}}$
同时可以把统计距离推广到更高维的地方。

要指出的是k距离并不只是和椭圆或圆联系在一起，任何距离测量只需符合以下特征：
d（P,Q）=d(Q,P)
d(P,Q)>0 如果P！＝Q
d(P,Q)=0,如果P=Q
d(P,Q)<=d(P,R)+d(R,Q) (三角不等式)

对于统计距离的解释可以根据以下定理： \inline$(x-\mu)\sum^{-1}(x-\mu)$ 为 \inline$\chi_{p}^{2}$ 分布，其中 \inline$\chi_{p}^{2}$ 为自由度为p的卡方分布，\inline$(x-\mu)\sum^{-1}(x-\mu)$ 为多元正态分布的形式

根据卡方分布的定义我们可以得出
\inline$(x-\mu)\sum^{-1}(x-\mu)$$=Z_{1}^{2}+Z_{2}^{2}+........+Z_{P}^{2}$

其中 \inline$Z=\sum^{-\frac{1}{2}}(X-\mu)$$是\inline$N_{p}(0,I_{p})$ 的正态分布

所以统计距离平方可以转化为p个 独立的标准正态分布变量的平方和，即把标准正态分布变量数值看成坐标值。其中 \inline$(x-\mu)\sum^{-1}(x-\mu)$ 也称作Mahalanobis距离（由P. C. Mahalanobisi在1936年提出）。Mahalanobis距离可以检测线性回归模型中的异常值。

统计距离在聚类分析中有着更广泛的应用，聚类分析有通过距离测度来获得相似性测度的方法，即通过统计距离的远近来判断案例（在坐标系中视作点）是否相似，属于同一类别（这也是统计距离的初衷所在）。所用的距离测度有：

欧式距离


（引自郭志刚《社会统计分析方法——SPSS软件应用》p122）


绝对值距离


（引自郭志刚《社会统计分析方法——SPSS软件应用》p122）

Minkowski距离

（引自郭志刚《社会统计分析方法——SPSS软件应用》p122）

Mahalanobis距离

（引自郭志刚《社会统计分析方法——SPSS软件应用》p122）


统计距离此外还有Cook距离 （http://en.wikipedia.org/wiki/Cook%27s_distance） 、Hellinger 距离 （http://en.wikipedia.org/wiki/Hellinger_distance）和Bhatta charyya距离 （http://en.wikipedia.org/wiki/Bhattacharyya_distance）等

参考文献
1.郭志刚（1999）郭志刚《社会统计分析方法——SPSS软件应用》，人民大学出版社
2.理查德　Ａ约翰逊，迪安　Ｗ威克恩（２00８）,实用多元统计分析（影印版），清华大学出版社
3.维基有关statistical distance的相关条目

chernoff face

今天读《实用多元统计分析》（“applied multivariate statistical analysis”）时，读到有关 chernoff face的介绍，觉得挺有意思。Chernoff face是一种有趣的数据呈现方法。它把多元数据用二维的人脸的方式整体表现出来。各类数据变量经过编码后，转变为脸型，眉毛，眼睛，鼻子，嘴，下巴等面部特征来，数据整体就是一张表情各异的人脸。Chernoff在７0年代提出了chernoff face的技术。他认为chernoff face可以用来表示最多为１８种变量的数据。Bernhard Flury和Hans Riedwy（１９８１）认为，保持chernoff face脸对称并没有意义，因此他们主张用另外１８个不同的数据点来表示另外半张脸（在我看来这为数据的分半信度提供了很好的方法）。所以chernof face可以表示３６个数据点。

Chernoff face是一个需要计算机才能实现的技术，很多软件提供了此类服务，比如有专门制作chernoff face的软件Faces 2.1（下载地址为http://bradandkathy.com/files/Faces2.zip）R中也有相应的代码，具体代码和使用可以看http://rss.acs.unt.edu/Rdoc/library/TeachingDemos/html/faces.html。

利用人对人脸识别的天生能力来鉴别数据的整体特征。这是一个有意思的创意。但是转化后的人脸意味着什么？我们如何把多元数据真正合理地类比为人脸，这不是一个容易回答的问题。最初的chernoff face中，变量编码为哪一个面部特征是随意的，这就使得数据呈现无法利用面部识别的特点（整体性），而且也很容易对数据的真正意义产生误解。Morris, Ebert和Rheingans等人（１９９９）对chernoff face的实际效果进行了实验研究，发现对被试chernoff face的识别是序列搜索任务，而不是发生在平行搜索的前注意阶段。chernoff faced的眼部表情在识别作用最大。但总体上chernoff face并不见得比其他数据呈现图形效果更好。同时chernoff face虽然有助于知觉，但是不同的面部特征之间可能会出现交互作用，因此对面部特征的知觉空间可能是非线性的。（http://eagereyes.org/VisCrit/ChernoffFaces.html）

最近有人提出了EVA(empathic visualization algorithm)的方法（http://www.cs.ucl.ac.uk/staff/a.loizides/218.pdf），对chernoff face进行了修正和扩展。EVA方法将语义上重要的数据变量编码为情绪上重要的面部特征（如脸的表情，肌肉紧张度等）。EVA强调数据的特征对面部特征起着决定作用，整体值对整体特征，缺失值对缺失特征，两者严格对应，这样的话chernoff face的面部特征就是数据的真实体现。然而要做到这一点，必须对数据进行预先的严格分析，这就要耗费比较大的精力。也许chernoff face只是一个说服别人的工具,而不是一个探索问题的工具。Chernoff face最重要的功能就是分类和校准。我们可以通过chernoff face很轻易地把一个不同于其他的数据样本挑出来。

对面部的精确识别，这是人的一个很奇妙的能力，也是心理学家试图探索而没有完整解决的问题。chernoff face应用的得与失，也许只有随着面部识别的研究进展才能更好的回答了。现在还是欣赏一下我找到的chernoff face吧：）

R中 关于chernoff face 的例子：


真人版chernoff　face：
(引自http://alexreisner.com/baseball/stats/chernoff)


参考文献
１．Wiki: chernoff face http://en.wikipedia.org/wiki/Chernoff_face
２．理查德　Ａ约翰逊，迪安　Ｗ威克恩（２00８）,实用多元统计分析（影印版），清华大学出版社
３．Morris CJ, Ebert DS, Rheingans（1999） "An Experimental Analysis of the Pre-Attentiveness of Features in Chernoff Faces", published by the conference "Applied Imagery Pattern Recognition: 3D Visualization for Data Exploration and Decision Making".

４．Social Science Statistics Blog: Chernoff face:
http://www.iq.harvard.edu/blog/sss/archives/2006/11/chernoff_faces_1.shtml

漏掉了的一篇笔记

今天在清点读书笔记时，有一篇记着的总是找不到，结果发现在转移阵地时不慎遗弃了。于是赶紧把它拯救回来。虽然说只是对一篇文章粗浅的读书笔记，引文方面还不是很规范，但终究还是自己在读书完的的一点劳动。所以，人民（特指我）不会忘记你！

今天随便翻阅了《the sage handbook of quantitative methods in the social sciences》中Mulaik写的"Objectivity in Science and Structural Equation Model" 一章，作者从康德关于先验范畴的讨论爬到对科学和结构方程客观性的意义，使我印象比较深的是作者把客观性看作是将非经验的概念有效化的方式，换句话说就是内部一致性共识，既然客观性概念不再是客观的，那么因果关系的客观性也更加值得怀疑。读书笔记中阅读的文章就是在因果关系的主观和客观的解释及理论中摇摆。对我有感悟的是activity theory of causality， 它先是把因果看成三个层面的概念，一是自由意志观，相信我们的意识能自由主导我们的行为，诸如历史上的英雄决定成败之类的；二是物理因果观，在初始条件设定的情况下以一种机械的固定不变的方式运作，像机械钟的运动；心理层面的因果和物理层面的因果由于心身二元论的对立而无法融合在一起，因此人们又发明了第三种因果——逻辑因果，诸如充分、必要条件之类的。虽然逻辑因果可以用逻辑和数学的方式进行精确、严密的处理，但逻辑和数学被Mulaik指出终究是经验处理规范的隐喻。逻辑因果把物理事实用隐喻的方式抽象出来在心灵层面操作，这依旧无法解决两者的对立。可以说因果的客观性是无法被有效证实的，而activity theory of causality 更强调它对现实生活的指导意义，也就是说，提出的因果的关系不能仅仅停留在逻辑上的严密性和完备性，它必须是现实世界中可以直接操作，并对我们世界和社会的改善有着积极的影响。我想，因果是否仅仅是思维的经济法则，通过最小成本（事件在时空上的最短距离的组合）获得最大收益（直接预测和控制对我们生活有最大影响的事件的发生）。

也许我有机会要结合凯恩斯的经历思考一下经济学的本质了：）


Causal Inference from Philosophical and Methodological Perspective
bv
As it is often said, every why has a wherefore. It is natural for us to assume every event has a cause. However, understanding the causal relationship in scientific research is a troublesome one where different schools explain the causal relationship according their own perspectives. One of the earliest persons talking about causal inference is Hume who argued that people see some cases as causal events according to three points. The first point is the contiguity between the presumed cause and effect, the second temporal precedence between them and the third their constant conjunction. An instance taken by Hume is many glass balls colliding with each other. He argued that it was difficult for us to distinguish the cause and effect in such complex situation where the only principle we may rely was the closeness of events in space and time. However, are causalities people believe just the coincidences in space and time? Hume reached his conclusion in a sarcastic tongue: correlation means causal relationship. In other words, the psychological illustrations induce people believe the causal relationships exist in the world. Although there are many inconsistencies in fundamental points with Hume, the positivist also denied the importance of objective causal relationship in theory. Russel, a representative of positivist, argued that since the mathematical functions have sufficiently explain the relationships of variables in nature, is it necessary to remain a causal interpretation as a attractive but not useful method. Besides, Russel pointed out: contrast to the symmetrical properties of mathematical and physical laws, causality is unidirectional. It is strange from the view of mathematicians While Russel’s points seem too extreme, he successfully makes us focus on a fact that role played by causal inference in natural science is less important than in social science. The causal inference may be a delicate alternative as a more powerful analyzing tool.

Contrast with positivist’s prejudice to causality, essentialist believe the existence of underlying causes which hide behind the observational phenomenon. They attempt to seek the micromediational mechanisms which will offer the ultimate interpretation. Their perspectives are associated with reductionism. As Campbell and Stanley argued that experiment can probe but not prove the causal hypothesis, positivist’s operational definitions and strict manipulations don’t reflect the true meaning of cause, a deeper exploration is needed.

What’s the true meaning of causality? Different persons provide their own answer. Mill emphasized the temporal order and relevance of cause and effect as the indicator to the causality. In order to exclude the alternative possibilities, the joint method of agreement and difference is designed by Mill, who convincingly proved its effectiveness. Popper insisted falsification is a key method to test the available causal hypothesis.

While Mill’s methods are applied widely in the experimental design to explore the causes, the activity theory of causation raises a question: how can we test the causal relationship through observation. There are three levels of senses of causes according to the interpretation of activity theory of causation. Firstly, it is caused by “the free and deliberate act of a conscious and responsible agent”, where the free selections of individuals dominate the causal effect. The explanations of causes for the historical figures’ behaviors conform to that perspective. The second may call the mechanical causality such as mechanical clock’s work. The processions of events are fixed by the natural principles. The previous one naturally produces the next one. The third is the logical causation which fits the essentialists’ perspectives. It considers the sufficient and necessary condition of causation. Differed from the essentialists and positivists’ points, causal mechanisms are not seen so important in activity theory. Without understanding the concrete ways that causes influence effects, people are still able to predict and control effect in virtue of causation. According to the activity theory, the causal interpretation makes sense only when the causal factors can be manipulated reliably. A cause unable to help us improve the world is no useful in theory.

The evolutionary critical-Realist perspective believes while causal relationships are objective and existing beyond human conscious, our imperfect organ and intellectual limits us to perceive the accurate causal relationship. In another words, our causal perceptions are the product of million years of evolution. Survival values and logical meaning, which is ultimate goal of causality? Evolutionary perspective emphasizes the former. They try to look for the origin of the conception of causality from the biological evolution which make the causality appears not so

Until now, experiment is still a dominating method to confirm the causality. According to the logic of experiment, causal effects will be appeared through the manipulation of independent variables and the setting of the alternatives in the laboratory. However, the strictness of logic limits the causal exploration to the simple and effective part compared with the attempt to build the big and whole causal chains, which will reduce the real attractiveness of causation as the our key to understand the world.

Below is the eight assumptions about causal chains from the Cook and Campbell's article (Cook, T.D., & Campbell, D.T. 1979) . I hope they will be useful for someone:

1.Causal assertion are meaningful at the molar level even when the ultimate micromediation is not known

2.Molar causal laws, because they are contingent on many other conditions and causal laws, are fallible and hence probabilistic.

3.The effects in molar causal laws can be the result of multiple causes.

4.While it is easiest for molar causal laws to be detected in closed systems with controlled condition，field research involve mostly open systems.

5.Dependentable intermediate mediational units are involved in most strong molar laws.

6.Effect follow causes in time, even though they may be instantaneous at the level of ultimate micro mediation.

7.Some causal laws can be reversed, with cause and effect interchangeable

8.The paradigmatic assertion in causal relationships is that manipulation of a cause will result in the manipulation of an effect.


reference:

Cook, T.D., & Campbell, D.T. (1979). Causal inference and the language of experimentation. Quasi-Experimentation: Design and Analysis Issues for Field Settings. pg. 1-36. Boston: Houghton Mifflin Co.

《女士饮茶》的读书笔记

最近一两天把李老师热情推荐的八卦书《女士饮茶》给读完了，虽然大多时候是当八卦读，但是感觉收获确实很多，至少从统计史的角度认识了统计学的意义。下面是一些自己感兴趣的片段的记录，算是纪念这本书的阅读历程吧。

奇人奇事

1．皮尔逊读的是政治学博士，研究的是科学哲学问题。这位统计学的奠基者创造了统计学，也创造了统计的哲学思考了吧。也许统计学就是对数据的哲学思考。从皮尔逊的身上我们可以看到统计学和政治学、科学哲学如何紧密联系起来的。

2．皮尔逊对头骨和骨骼的统计学分析用来解决历史上的难题。

3．从达尔文到高尔顿我们可以看到生物学与行为科学对早期统计的影响。回归趋势、正态分布、相关，这些都是高尔顿的贡献。我在想生物学为基础的统计和政治社会学为基础的统计如何融合为一起的，从生物进化论到社会进化论？这种影响对日后的统计学发展又会有什么影响？

4．《生物统计》是高尔顿创立的第一个重要的统计杂志，高尔顿的经费是如此充足，以至于杂志都为全彩照片，这让我想到自己刚看到《科学·科学美国人中文版》时的感受了，不知道《生物统计》那时侯售价多少，个人是否订的起？现在拍卖的价格又是多少？

5．戈塞特用自己化学的背景进入酿酒企业，用自己的数学背景研究统计学问题，随后他的统计学研究文章成了企业重要资产，他还升上了企业管理层，真是一个工作、研究两不误的好典型。T-分布起源于戈塞特以学生（student）为笔名发表的文章，戈塞特当年的谨慎弄的我们现在刚学统计时对t-分布的意思摸不着头脑。

6．费歇尔，务过农，当过兵、教过书，就差没做过工了。他的近视造成他极强几何直觉能力，但他的几何直觉能力又让他不被人理解，塞翁失马，焉知非福。

7．费歇尔从农田里收获了统计学的果实：随机化、方差分析、自由度。费歇尔大概用统计学描述了农业收成的谱写了一曲华丽的乐章。

8．世态炎凉：艾森哈特想去拜访老年的皮尔逊，被同事所阻止：看那个老家伙有什么收获？于是那个老家伙只能孤零零地住在远离两个系（优生学系和生物统计学系）和生物统计研究所的办公室，庆幸的是他的儿子当了生物统计系的系主任，只是不知道英国人是否有中国人的想法。

9．皮尔逊的代表作：《科学的法则》 费歇尔的代表作：《研究工作者的统计方法》

10．布利斯研究杀虫剂提出概率单位分析，在列宁格勒布利斯和克格勃的对话让我感到做一个单纯的人真好。度过了大清洗的残酷和卫国战争的血腥，布利斯迎来了温馨的秋季，秋季虽然美好但很多人和事都凋零了。

11．奈曼通过勒贝格的书进入了数学殿堂，可是亲身接触勒贝格时被他的傲慢所伤害了。后来为了弥补所带来的伤害，奈曼成了平易近人的老师。奈曼把文章写的非常的简单、自然，这是学术文章的最高境界吧。

趣事：奈曼在国际会议上宣讲一篇法语文章时，已经做好准备迎接费歇尔刻薄而强力的批评，结果费歇尔表现的非常平静，原因是他不会讲法语。

12．柯尔莫戈洛夫，一个天才加全才的数学家，他在莫斯科大学数学系从没考过试，原因是他写出了14篇独创性的论文来代替14门基础课的考试（当然他后来承认有一篇的结果其实是错的）。他70多岁的时候还爬山、海洋探险、滑雪、和教皇讨论宗教史，背普希金的诗歌。

13．饼图是护士行业的传奇女性南丁格尔发明的，大概是为了和愚蠢无知的军事将领打交道时更好的说明自己的观点吧。她的女儿大卫是皮尔逊晚年的研究生，不过相当怕皮尔逊。因为费歇尔对女生不屑一顾，所以大卫想向费歇尔提问时总是请旁边的男生代劳。大卫后来写了一本组合数学，很经典的著作。

14．皮特曼解决了非参数检验中关于检验效力和检验范围的问题，他是一个墨尔本大学的数学本科学生，但当他担任统计学教授，他还没正式接触过统计学的理论。

15．大萧条中大学生找不到合适的工作，结果他们去了劳工部和商务部，从实际问题中提出了抽样理论。

16．斯内德克建立了美国第一个统计系。

17．古德说，2的开方是无理数的发现“如果是当今的大人物所为，我会觉得很平常，但在两千五百年前是个惊人之举”，因为他在10岁时就通过心算发现了这一点。古德对偶然出现的数字巧合很感兴趣，所以后来他的书有了哲学的意味。

18．迪亚科尼斯14岁时离家出走跟着一个魔术师学魔术，24岁时回来开始念纽约市立学院的成人教育班。他为了看懂一本研究生概率数学教材而想上大学。

19．博克斯因为看过费歇尔《研究工作者的统计方法》，所以被军方送去学统计。战后他报考伦敦大学的统计学研究生时，向皮尔逊大谈费歇尔的理论，结果皮尔逊静静听完后同意了他的申请，但希望他会知道统计界除了费歇尔还有别人存在。（皮尔逊真是一个厚道人！）后来博克斯还娶了费歇尔的女儿。

20．戴明提出用统计的方法进行质量的管理，启发了日本产业界人士，结果重塑了日本产业界，促进了日本经济的崛起。可惜的是墙内开花墙外香美国人很长时间不买他的帐。戴明嘲笑假设检验的广泛应用，提出有意义的是差异大小程度而不是差异显著性水平。

统计技术思想

1．费歇尔关于吸烟是否制癌的讨论，开始了统计学中什么是因果关系的讨论。实验设计能否揭示出因果关系。

2．皮尔逊用分布的概念重塑了人们对世界的理解，世界不再是精确的公式所简单描述的，而是随机图景中的概率分布。

分布的四个参数：平均数、标准差、对称性、峰度

如果一个分布接近正态分布，它只需考虑前两个参数：平均数和标准差

3．皮尔逊和皮尔逊个人矛盾演化为两种统计学派的矛盾，皮尔逊认为统计分布是真实分布的描述，费歇尔认为统计分布只是用来估计真实分布的参数。皮尔逊用假设检验，费歇尔用显著性检验。“知道如何设计实验，这个实验就几乎一定能给出一个显著性的结果”

4．中心极限定理：一个被广泛应用但到20世纪30年代还未证明的定理。

要证明中心极限定义，必须先证明符合Lindeberg-Levy 条件，要证明符合Llindeberg-Levy条件，先证明是U-统计量（可惜的是书中还是没告诉我什么是Lindeberg-Levy条件和U-统计量）也许这就是八卦书的无奈吧。

5．拟合度检验：混沌理论的一个缺陷：没有给出数据绘出的图形和混沌理论预测图形的拟合度，拟合度是皮尔逊的贡献，他最早提出了卡方检验。不过费歇尔批评他比较两种比例时得出的参数值弄错了。

6．概率论，柯尔莫戈洛夫解决了概率论的基础，把求解概率和求一个不规则图形的面积联系起来，把概率论和数学测试理论，随机过程联系起来。（逆命题我曾经在一本书中见到，如果要求一个不规则图形的面积，就把分布在图形内的随机点的数量和另一个已知面积的规则图形中的随机点的数量进行比例关系）

7．极大似然法，费歇尔提出，用迭代算法计算来逼近一致而有效的统计量。（书中就到这里，我还是不能理解，于是又查了一本比较普及的书极大似然法就是让概率密度函数的值尽可能最大化，用来使数据尽可能的符合特定的参数估计。具体怎么计算等我好好看百科全书吧：））

8．置信区间上奈曼提出的，置信区间中提出的概率是统计学家使用某种方法从长期以来看作出正确陈述的频率。

9．非参数检验：从几何的角度看就是将观测数据的散点图和纯随机分布的预期图形进行比较。

10．贝叶斯发现条件概率的公式是内部对称的，那么条件概率公式倒过来代表什么？用分布估计参数变到由参数估计分布？贝叶斯学派的两个取向：通过积累数据获得分布的信息，主观概率（先验概率和后验概率）

11．快速傅立叶变换：一种以电脑为基础的数据分析方法，可以用来分析一长串相互关联因素的影响结果，一种向临近频率借力的方法。

12．鞅（martingale）：如果数列满足变异是有界的和下一个数字的最佳估计值是他的前一个数值时，这样的数列称作鞅。鞅这类数列将趋向于正态分布。如果能把长时间得到的数据看成是鞅，那么我们就可以解决生存分析之类的问题。

13．格利文科－坎泰利引理：如果有一些数，我们对它们的概率一无所知，那么我们可以构造一个非参数分布，尽管构造的数学函数结构不雅观，我们还是可以通过增大观测值的数量使经验分布函数越来越逼近真实分布函数。

14．Bootstrap:通过对自身样本进行有放回的全抽样，从而提升数据自身的模拟过程。

核密度回归估计（kernel density-based regression estimation）：一种有关运算密集（在数学上对重复抽样方法的推广）的程序，由两个参数确定：核（kernel）和带宽（bandwidth）

统计与世界观

1．统计革命的基本观点：科学真实的主体是数字的分布，这个分布可以通过参数来描述。

2．在当今统计应用急剧发展的世界，统计学家已经失去对统计的控制权，各种各样的统计方法是从在应用行业使用统计的人手中直接得出。

3．坎利夫女士的忠告：人类本身在偏爱方面充满了变异，在产品制造和质量管理等方面要要注意这些变异。统计学家的工作就是阅读数据，并质疑它为什么这样。将数学问题尽可能的用数学模型的方式表达出来，会促进科学家充分了解会出现什么问题。“如果我们一本正经地对一个不懂统计的男人或女人说“P 值小于0.001”意味着什么，我们就不会成功，所以，我们必须用他们的语言来解释我们的发现，以增强说服工作的效力。”

统计学的三个哲学问题：

4．可以用统计模型做决策么？L·乔纳森·科恩的彩票悖论和无票入场者悖论

5．当概率应用于现实社会时，其含义是什么？

柯尔莫哥洛将概率定义为一个抽象空间里对一事件集合的一种测量。但我们如何来确定现实中的抽象空间。威廉·S·戈塞特试图为一个设计好的试验描述其事件空间。而第二种方法是用样本调查理论来选择抽样的方法来尽可能的使样本反映总体。

6．人们真的懂得什么是概率么？

凯恩斯在博士论文《关于概率的讨论》中指出概率的含义的结论取决于人类对不确定性量化的愿望，是一个和人的主观判断和文化背景有关的概念。（牛人的一篇牛文，有想看的冲动）凯恩斯的观念是进行统计决策的基础，影响到了卡尼曼等人关于主观概率和启发式的研究（李老师推荐了很多他的文章）

causal faithfulness and d-separation

SEM and Neyman-Rubin potential-outcome model are considered as two major statistical tools for explaining the causal relationships of two variables. However, as Pearl（1997） says,SEM is used by many people and understood by few, while potential-outcome model is understood by few and used by fewer. People apply SEMs to solve the problems in their academic field without guaranteeing their power of causal interpretation. Therefore, as Pearl points out, “ the current dominating philosophy treats SEM as just a convenient way to encode density functions (in economics) or covariance information (in social science). ” So John Fox introduces a cynical view of SEMs in the appendix on structural equation model of his book（http://socserv.mcmaster.ca/jfox/Books/Companion/appendix-sems.pdf） that the popularity of SEMs may be attributed to their pretentious ability of causal interpretation of observational data although they’re not less problematic than other regression models, or they just translate the informal thinking of causal relationships into a formal data analysis.

However, Pearl（2000） believes there is causal interpretations for observational data from SEMs. He argues that almost all the academic educations and publications overlook the presuming condition raised by the fathers of SEM, that is in a equation like y=βx + ε, if we make sure that it is structural, β should be the unique causal connection between x and y, the statistical relationships of x and εcan not be changed because of the different interpretation of β. That is the condition called self-containment. In the language of graph favored by Pearl, it is also named d-separation.

d-separation is clarified by Pearl in his blog article “d-Separation Without Tears” （http://www.mii.ucla.edu/causality/?m=200001） as a criteria to determine whether a set X of variables is independent of a set Y of variables given a third set Z. If X and Y satisfy the conditions of d-separation, we can say X and Y is d-separated or disconnected. If we use the language of graph, x and y is d-connected if there is an unblocked path between them. In the graph \inline $x\rightarrow m\rightarrow o\rightarrow n\rightarrow y$ , x and y is d-connected conditioned on the set of Z, if X and Y have no collider-tree path that traverses the member of Z, we can say X and Y is d-separated by Z. if r and v are the members of Z like graph \inline $x\rightarrow m'\rightarrow o\rightarrow n'\rightarrow y$ , X and Y is d-separated by Z. however, if a collider is the member of conditioning set Z, like m’ and v’ in the paragraph below, s and y is d-connected because t has p’ as the member of z.

（quoted from Pearl’s blog article “d-Separation Without Tears” http://www.mii.ucla.edu/causality/?m=200001)

In another words, x and y is d-separated by z means there is the null partial correlation between x and y by z.

D-separation is the a key assumption for the causal faithfulness, which promises every conditional independence of variable in the causal relationship so that the causal structure can be testable. Therefore, d-separation is used to make the causal interpretation of SEMs more reliable. If we test the causality of structural equation model, we should carefully test the self-containment of equation by using d-separation to compute the conditional independence relations. Now I just primarily understand the conception of d-separation. A lot of efforts are still needed for me to study the statistical reading of causality by applying d-separation.

Reference:
d-separation, http://www.andrew.cmu.edu/user/scheines/tutor/d-sep.html#d-sepapplet2

Pearl’s blog article “d-Separation Without Tears” January 1, 2000

Judea Pearl (2000), Causality: Models of Reasoning and Inference, Cambridge University Press

John Fox, Web appendix to “An R and S-PLUS Companion to Applied Regression”, Structural-Equation Models http://socserv.mcmaster.ca/jfox/Books/Companion/appendix-sems.pdf

Pearl(1997), The New Challenge: From a Century of Statistics to the Age of Causation，Computing Science and Statistics，29，415--423

The interpretation of Lord’s paradox in Rubin causal model

Lord’s paradox was first raised by Lord in an article on “Psychological Bulletin” in 1967. It reveals a contrast of two statisticians’ conclusions based on the same set of data. Here are the Lord’s four examples to illustrate his points.

Example1: “A large university is interested in investigating the effects on the students of the diet provided in the university dining halls and any sex differences in these effects. Various types of data are gathered. In particular, the weight of each student at the time of his arrival in September and his weight the following June are recorded. ”(Lord,1967)

Example 2: “A group of underprivileged students is to be compared with a control group on freshman grade-point average than the control group. However, the underprivileged group started with a considerably lower mean aptitude score(x) than did the control group. Is the observed difference between groups on y attributable to initial differences on x? or shall we conclude that the two groups achieve differently even after allowing for initial differences in measured aptitude” (lord,1969)

Example 3 “suppose an agronomist is studying the yield of various varieties of corn. He plants 20 flower pots with seeds of a “white” variety. For simplicity of illustration. Suppose that he treat all 40 plants equally for several months, after which he finds that the white variety has yielded considerably more marketable grain than the black variety. However, it is a fact that black variety plants average only 6 feet high at flowering time: whereas white variety plants average 7 feet. He now asks the question, would the black variety process as much salable grain if conditions were adjusted so that it averaged 7 feet in height at flowering time?” ( Lord,1969)

Example 4: “consider the problem of evaluating federally funded special education programs. A group of disadvantaged children are pretested in September, then enrolled in a special program, and finally posttested in June. A control group of children are similarly pretested and posttested but not enrolled in the special program. Since the most disadvantaged children are selected for the special program, the control group will typically have higher pretest scores than the disadvantaged group”(Lord,1973)

Through analyzing four examples, it is easily found that the previous inconsistence on conditions of experimental group and control group make the statistical hypothesis difficult to balance them. Therefore, it is necessary to apply counterfactual thinking to modify the previous condition in theory. However, the question is whether it is available to such modifications.

Rubin(1982) investigated the Lord's paradox in the form of Rubin causal model. The first statistician assumes the causal effects in a form below.
\inline \emph{$D_{i}=E(Y_{t}-X\lyxmathsym{\textSFxi}G=i),$ i=1,2}
\inline \emph{$D=D_{1}-D_{2}$}
（quoted from Holland, Paul W. Rubin, Donald B,1982)

E represents the experimental effect, \inline \emph{$Y_{t}$ } represents the outcome variable in the experimental group, G represents the subpopulation indicator variable, X represents the concomitant variable. D represents causal effect.

The first statistician finds that there is no difference between the concomitant variable and outcome variable for both females and males.

Considering the previously differences of experimental group and control group before the treatment and attempting to use covariance to control it , the second statistician computes the causal effect in such a way below:
\inline\emph{$D_{i}=E(Y_{t}-X\lyxmathsym{\textSFxi}G=i),$ i=1,2}
\inline \emph{$D=D_{1}-D_{2}$}
(quoted from Holland, Paul W. Rubin, Donald B,1982)

If we convert the equation of causal effect into a regression form, we can get the equations below:
\inline \emph{$D_{i}=E(Y_{t}-X\lyxmathsym{\textSFxi}X,G=i)=a_{i}+bX$ i=1,2,}
\inline \emph{$D_{i}(x)=a_{i}+(b-1)x,$ i=1,2,}
(quoted from Holland, Paul W. Rubin, Donald B,1982)

In the equation, \inline $a_{i}$ represents the differences of previous conditions on experimental group and control group; b represents the influence of the causal effect in the experimental group. In Lord’s paradox, two statisticians have different underlying assumption on \inline $a_{i}$ and b. The first statistician assumes \inline $a_{i}$ is o and b is 1, while the second assumes b is same in two groups. Therefore, they make their conclusions according their different underlying assumptions.

From the view of Rubin causal model, there is a concomitant variable (X) closely associated with the outcome variable (Y) in the example of Lord’s paradox. The concomitant variables produce the previously unequal conditions and affect the following outcome variables.

Rubin pointed out that there are some underlying assumptions in the statistical hypothesis the statistician made such as which is not testified appropriately. These assumptions have influenced the conclusions the statisticians reach

Rubin identify three types of studies as descriptive studies, uncontrolled causal studies and the controlled causal studies. The descriptive studies have no experimental manipulation. The uncontrolled causal studies have experimental manipulations without strictly controlling relevant factors. All possible factors have been sufficiently considered by experimenter in the controlled causal studies. The first statistician uses the unconditional descriptive statement that control group and experimental group are equal before treatment. The second statistician uses conditional statement which considers the previous differences. If both statisticians use descriptive statements, they are both right. However, when the descriptive statements are converted into causal statement, neither of them is right.

If I apply the Rubin and Holland’s analysis to the example of Lord’s paradox in the Powerpoint （http://lixiaoxu.googlepages.com/08Dec2006.ALL.G.ppt）, I can assume P represents the students in the class; t represents the course students receive; G represents the genders of students; X represents the degree of confidence before the course; Y represents the degree of the confidence after the course. Then we can see two statisticians make the different conclusions. Because the first assumes average confidence gains for males and females are equal, and the second assumes the male students and female students have equal confidence before course. We can see that two statisticians respectively make the untestified underlying assumptions before raise their null hypothesis. Those two counterfactual thinking both go against the real causal chains. It is doubtful to simply alter the causal factor without careful consideration. It seems to be a paradox for me to setting the experimental condition according the hypothesized causal relationship to explore the real causal relationships.

Lord’s paradox reflects the influence of differences of statistical hypothesis on the ultimate conclusion made. Can statistical hypothesis recognize and eliminate the previously existed unequality? In another words, descriptive statement and causal statements are two different language systems. The free translation between them seems not a certain thing. Since hypothesis test can only fix two contrast propositions, it is necessary for us to understand the limits of statistical language.

Reference

Holland, Paul W. Rubin, Donald B.(1982), On Lord's Paradox. Program Statistics Research

Lord, F. M.(1969) Statistical adjustments when comparing preexisting groups. Pwchological. Bulletin, 72, 336- 337

Lord, F. M.(1967) A paradox in the interpretation of group comparisons. Psychological Bulletin, 68 , 304-305

causality and statistics

In methodology and philosophy, causality has been discussed by experts for a long time. However, in statistics, the avoidance to talk causality became a common trend among early statisticians before 1970s. Since correlation doesn’t equal to causality, the interpretations of causality in statistics have been doubted in academic fields. Because of the wide application of experimental methods in scientific research, especially for the establishment of well-designed randomized experiments, has induced statisticians to seek for the statistical form of causality. Rubin developed the first causal model for analyzing the relationship between cause and effect. He hypothesized that the potential outcomes would be produced through the manipulation of experimental conditions. By comparing the outcomes for the same unit, we will measure the causal effect. The framework of Rubin causal model has been constructed through the introduction of the concept “counterfactual assumption” which means we assume how the world will behave if event X is absent. The concept “counterfactual assumption” reminds me of an article named “culture and cause: American and Chinese attributions for social and physical events” by Kaiping Peng and Morris (1994) . In one study of the article, they applied the counterfactual fact judgment to investigate the causal inferences of Americans and Chinese on the murderer Lu Gang. Nowadays counterfactual assumption has been a powerful tool in the research of causality, but its limitations have been revealed because of Lord’s paradox which will be discussed in next article.

By contrast to studying the cause of give effect, Rubin pay his attention to the causal effect, which is defined by him below:

“intuitively, the causal effect of one treatment, E, over another, C, for a particular unit and an interval of time from t₁ to t₂ is the difference between what would have happened at time t₂ if the unit had been exposed to E initiated at t₁ and what would have happened at t₂ if the unit had been exposed to C initiated at t₁: 'If an hour ago I had taken two aspirins instead of just a glass of water, my headache would now be gone,' or because an hour ago I took two aspirins instead of just a glass of water, my headache is now gone.' Our definition of the causal effect of the E versus C treatment will reflect this intuitive meaning.” (Rubin, Donald B., 1974)

According to the definition, Rubin (1982) applied some symbols in his analysis of causal effects to represent the elements of causal effect. Among them P represents the population of unit; t represents experimental manipulation on different levels; c represents experimental manipulation on level t; S represents the associated indicator of experimental manipulation; G represents a subpopulation indicator variable; X represents a concomitant variable; Y represents an outcome variable. In terms of those symbols, Rubin can discuss the causal effect in more accurate mathematical and statistical language.

In Rubin causal model, several assumptions should be noticed, which are temporal stability, causal transience, unit homogeneity, independence and the constant effect (Holland, 1986) . Temporal stability means the causal effect of treatment doesn’t depend on when we start the treatment, that is to say the response of unit is constant through the time. Causal transience means the causal effect wouldn’t last long to affect the measurement next time. Unit homogeneity plays a important role since it constructs the validity of experiment in laboratory. It assumes causal effect happen in one unit can produce a same effect in other units. Without that assumption, it is impossible for us to convince the outcomes acquired from the laboratories. However, unit homogeneity assumption is never testified effectively while it creates a necessary condition to make experiment reliable. There is a close association between the assumption of independence and randomization. The assumption of independence means the cause we select to be exposed is independent of other variables through the correct randomization. The constant effect means the effect on every unit is same, which makes the additivity of experimental effect possible.

Those assumptions are necessary for the solution of fundamental problem of causal inference, which is “It is impossible to observe the value of \inline $Y_{t}(u)$ and \inline $Y_{c}(u)$ on the same unit and, therefore, it is impossible to observe the effect of t on u” (Holland, 1986) . The possibility of causal inference would be endangered by the fundamental problem. There are two solutions for the fundamental problem of causal inference: scientific solution and statistical solution. The scientific solution is based on the homogeneity or invariance assumption, which makes scientists believe the causal effects happened at different times are same while the assumption is never proved. The statistical solution is based on the concept of average causal effect, which extends the causal effect on a specific unit to the general unit population and can be calculated by statistical method.

As we can see, time plays an important role in the causal model, the equality of the same causal effects happened at different times guarantees the effectiveness of causal effects. Time will eliminate the causal effects produced previously and create conditions for next one. Time will change the units without experimental treatment. So how to separate the interference of time from the causal inference is a serious problem.

From the view of Rubin causal model and its extension, cause can only be revealed from the treatment in experiment, where experiment can be generalized as any manipulation to the event. In article “causal inference and statistics”, Holland believes that the concepts of causality gotten from the operation in experiment and research in observational research are the same. (It makes me produce a question that whether thought experiment is a experiment). He compared three use of the word “cause”: the attributes one has possesses, voluntary activity and activity that is imposed on one. The first can not be the cause in experiment. The author wrote a short but meaningful sentence: every thing has a cause (law of causality), but not every thing can be a cause.

In Rubin causal model the causal inference is discussed within the experimental framework. It is an advantage, however, in my opinion, is also a limit, for it is difficult to interpret the causal relationship in non-experimental situation. I have doubts on whether experiment (even general one) can embody the entire causal relationships we talk. I remember Andrew Gelman said in his article "Foreign aid and military intervention, or Statistical modeling, causal inference, and social science" in his blog, " To me, a laboratory evokes images of test tubes and scientific experiments, whereas for me (and, I think, for most quantitative social scientists), the world is something that we gather data on and learn about rather than directly manipulate."

Reference

Holland (1986), causal inference and statistics, Journal of the American Statistical Association, Vol. 81, No. 396 pp. 945- 960

the article about Rubin causal model in wikipedia

http://en.wikipedia.org/wiki/Rubin_causal_model

Holland, Paul W. Rubin, Donald B.(1982), On Lord's Paradox. Program Statistics Research

Morris M W,Peng K P.(1994) Culture and cause:American and Chinese attributions for social and physical events.Journal of Personality and Social Psychology, 67, 949-971

Rubin, Donald B. (1974) Estimating causal effects of treatments in randomized and non-randomized studies . Journal of Educational Psychology 66:688-701