DDA3020 Homework 1 Due date: March 09, 2025 Instructions • The deadline is 23:59, March 09, 2025. • The weight of this assignment in the final grade is 20%. • Electronic submission: Turn in solutions electronically via Blackboard. Be sure to submit your answers as one pdf file plus two python scripts for programming questions. Please name your solution files as“DDA3020HW1 studentID name.pdf”, “HW1 name Q1.ipynb” and “HW1 name Q2.ipynb” (“.py” files are also acceptable). • The complete and executable codes must be submitted. If you only fill in some of the results in your answer report for programming questions and do not submit the source code (.py or .ipynb files), you will receive 0 points for the question. • Note that late submissions will result in discounted scores: 0-48 hours → 50%, more hours → 0%. • Answer the questions in English. Otherwise, you’ll lose half of the points. • Collaboration policy: You need to solve all questions independently and collaboration between students is NOT allowed. 1 Written Problems (50 points) 1.1. (Closed-Form Solution, 25 points) Given a ridge regression with data X ∈ R n×m, y ∈ R n , where n is the number of data, m is the number of attributes used for prediction, under the assumption that data X is centered, i.e., x¯ = n 1 P n i=1 xi = 0, show that the closed-form solution of the cost function J(w, w0) = (y − Xw − w01) T (y − Xw − w01) + λwT w is wˆ0 = y¯ w = (XT X + λI) −1XT y 1 DDA3020 Machine Learning Spring 2025, CUHKSZ 1.2. (Support Vector Machine, 25 points) Consider a dataset with 2 points in 1D: (x1 = 0, y1 = −1) and (x2 = 1, y2 = 1). Consider mapping each point to 3D using the feature vector ϕ(x) = [1, √ 2x, x2 ] ⊤. (This is equivalent to using a second-order polynomial kernel.) The max margin classifier has the form: min ∥w∥ 2 s.t. y1 wT ϕ(x1) + w0 ≥ 1 y2 wT ϕ(x2) + w0 ≥ 1 (i) Write down a vector that is parallel to the optimal vector w. (ii) What is the value of the margin achieved by this w? Hint: recall that the margin is the distance from each support vector to the decision boundary. (iii) Solve for w, using the fact that the margin is equal to 1/∥w∥. (iv) Solve for w0 using your value for w and the optimization problem above. Hint: the points will be on the decision boundary,代写DDA3020 Machine Learning so the inequalities will be tight, think about the geometry of 2 points in space, with a line separating one from the other. (v) Write down the discriminant function f(x) = w0 + w⊤ϕ(x) as an explicit function of x. 2 Programming (50 points) 2.1. (Logistic Regression, 25 points) In this question, we will implement multi-class logistic regression using gradient descent. Follow code template provided in logistic assignment1.ipynb to solve this question. Noticably, this question should not be solved by using sklearn package, aiming to promote your understanding on gradient descent. A mathematical derivation of gradient is required in this problem. Provide your solution in your written answers. 2.2. (Support Vector Machine, 25 points) In this question, we will explore the use of Support Vector Machines (SVM) for both linear and non-linear tasks using the sklearn library, as outlined in the SVM assignment1.ipynb file. By following the instructions in the notebook, we will implement both types of SVMs to gain a foundational understanding of how to apply SVMs.
