Multivariate polynomial regression python github
Multivariate Polynomial Regression using gradient descent. In this assignment, polynomial regression models of degrees 1,2,3,4,5,6 have been developed for the 3D Road Network (North Jutland, Denmark) Data Set using gradient
descent method. R squared, RMSE and Squared Error values have been calculated and compared for each model to find the models which best fit the data, as well as ones which overfit the data. L1 and L2 regularisation has been implemented to explore the effect of regularisation on testing loss and overfitting. The first attribute(OSM_ID) has been dropped. LONGITUDE and LATITUDE values have been used to predict the target variable, ALTITUDE. The code is divided into two files, generate_polynomials.py and polynomial_regression.py. The generate_polynomials.py file will calculate the terms
Gradient DescentFor each model, the training error was plotted for each iteration. It is clear that the error drops with each iteration. The following figure shows the plot of training error for degree 3 model R Squared, RMSE and Squared-errorR Squared and RMSE was computed for each model. It follows that up till degree 3, the testing error drops with increasing degree, but increasing degree there after results in an increase in error. This suggests that the degree 3 model best fits the data, where as models of degree 4, 5 and 6 are overfitting the data. The increasing average absolute values of weights with increasing degree also suggests that the weights are assuming arbitrarily large values to fit the data. RegularisationTo address the problem of overfitting, L1 and L2 regularisation has been implemented for the degree 6 model. The following figures show the effect of regularisation on testing error. Regularisation results in a sharp decrease in testing error. In fact, the loss for degree 6 polynomial model with regularisation is comparable with the loss for degree 1,2,3 and 4 polynomial models without regularisation. Average absolute weight decreases sharply for the models with regularisation. Once regularised, the ws aren’t assuming large values to cause the model to oscillate wildly and overfit the data. Instructions for executing:Run # multivariate-polynomial-regressionHere are 2 public repositories matching this topic...
Multivariate Polynomial Regression using gradient descent with regularisation
Evaluation of multivariate Bernstein polynomials
Improve this pageAdd a description, image, and links to the multivariate-polynomial-regression topic page so that developers can more easily learn about it. Curate this topic Add this topic to your repoTo associate your repository with the multivariate-polynomial-regression topic, visit your repo's landing page and select "manage topics." Learn more |