COMS 4721: Review of prerequisites

The d-dimensional Euclidean space Rd is the d-dimensional vector space over the real numbers R where we have the familiar notions of distances and angles from classical plane geometry. The space Rd is comprised of vectors (or points), which can be added (z := x + y for x,y ∈ Rd) and scaled by real numbers (z := cx for c ∈ R and x ∈ Rd) to obtain other vectors in Rd. Every vector x ∈ Rd has a (Euclidean) length (or norm; also called the l2 norm), which is denoted by ‖x‖2. The length of the scaled vector cx for c ∈ R and x ∈ Rd is ‖cx‖2 = |c|‖x‖2. Vectors with length one are called unit vectors, and there is a unique vector of length zero which is the zero vector (or origin) 0. The Euclidean norm comes from the inner product (or dot product) 〈x,y〉 between vectors x,y ∈ Rd, which is defined to be the product of (i) the length of x, (ii) the length of y, and (iii) the cosine of the angle between x and y. (Sometimes we will also write the inner product as x>y.) Since the cosine of the angle between x and itself is 1, we have 〈x,x〉 = ‖x‖2. We say x and y are orthogonal if 〈x,y〉 = 0. Using arguments from plane geometry, it can be shown that the inner product is symmetric (〈x,y〉 = 〈y,x〉) and linear in its first argument (so 〈cx+ y, z〉 = c〈x, z〉+ 〈y, z〉). The distance dist(x,y) between vectors x and y is measured by the Euclidean norm of their difference, ‖x−y‖2. This is a metric (called the Euclidean metric or l2 metric): for all x,y, z ∈ Rd: • ‖x− y‖2 ≥ 0, and ‖x− y‖2 = 0 if and only if x = y; • ‖x− y‖2 = ‖y − x‖2 (symmetry); • ‖x− y‖2 ≤ ‖x− z‖2 + ‖y − z‖2 (triangle inequality). The triangle inequality is equivalent to ‖x+ y‖2 ≤ ‖x‖2 + ‖y‖2. To see why it holds, observe that ‖x+y‖2 = 〈x+y,x+y〉 = ‖x‖2+2〈x,y〉+‖y‖2 ≤ ‖x‖2+2‖x‖2‖y‖2+‖y‖2 = ( ‖x‖2 + ‖y‖2 )2 .