Regularization in Machine Learning


NumPy is the go-to library in Python for scientific computing. It supercharges your ability to work with numbers!

[{"selector":"#anim-5c1fd5ef-1632-4fde-af4d-cbc76b05c2c3","keyframes":{"opacity":[0,1]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-7d5b2dc3-fed8-4e3f-ac4c-9559a29780d0","keyframes":{"transform":["translate3d(0px, 155.64701%, 0)","translate3d(0px, 0px, 0)"]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-c9d04083-3441-4b04-ab88-63ea9b86cc10","keyframes":{"opacity":[0,1]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-84ea850b-fb22-41a1-beb2-d5ab4a0b3077","keyframes":{"transform":["translate3d(0px, 227.94594%, 0)","translate3d(0px, 0px, 0)"]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-eee1b1ae-137c-4583-b066-086bbd99ac24","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-9ef59dcc-deef-4f73-a712-7129d98af93b","keyframes":{"transform":["translate3d(0px, -197.58459%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] Read More

Overfitting: Model learns training data too well, poor performance on new data. Underfitting: Model too simple. Regularization helps.

[{"selector":"#anim-a82a54b6-49f2-4c80-9422-5cf6c499692f","keyframes":{"opacity":[0,1]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-862637e8-e037-4cbe-98a3-95881fa7bd9f","keyframes":{"transform":["translate3d(0px, 139.42653%, 0)","translate3d(0px, 0px, 0)"]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-c3663003-c690-49cc-824e-3a534f05a4eb","keyframes":{"opacity":[0,1]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-4ec3a074-ddad-436d-95d5-38ade13baa41","keyframes":{"transform":["translate3d(0px, 227.94594%, 0)","translate3d(0px, 0px, 0)"]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-00bb28a4-d1fd-49ee-bb6e-24b7092665c0","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-28bccc0a-76d0-4dd7-8f2a-4ed353b89896","keyframes":{"transform":["translate3d(0px, -245.74314%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] Read More

It adds absolute value of coefficients to loss. Encourages sparsity, feature selection.

[{"selector":"#anim-160105fb-07e1-4a65-a5a0-3ecd8f89e5a8","keyframes":{"opacity":[0,1]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-ed1ce620-8235-49ee-ad6d-1c80d6c8eb22","keyframes":{"transform":["translate3d(0px, 210.47254%, 0)","translate3d(0px, 0px, 0)"]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-495bf52e-729e-4c18-a82b-216cc48ac962","keyframes":{"opacity":[0,1]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-ed4cf9ae-1b6d-4c09-baa2-9cf3802d27bd","keyframes":{"transform":["translate3d(0px, 227.94594%, 0)","translate3d(0px, 0px, 0)"]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-ed89a946-ac32-4976-a2de-102cbd4b4b6b","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-5bedd364-6692-4a01-a7a4-b2e508ec6687","keyframes":{"transform":["translate3d(0px, -245.74314%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] Read More

It adds a square of coefficients to loss. Shrinks coefficients towards zero, prevents overfitting.

[{"selector":"#anim-ecab227b-6d9d-4c59-ab98-c00cd8c61975","keyframes":{"opacity":[0,1]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-b09ccae3-240c-4811-917f-d25628d64240","keyframes":{"transform":["translate3d(0px, 209.26986%, 0)","translate3d(0px, 0px, 0)"]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-0ab3f645-31b5-4c5a-b7e9-a66508bf21ea","keyframes":{"opacity":[0,1]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-736c4561-ae5f-4f5b-a1da-371a6a6d2410","keyframes":{"transform":["translate3d(0px, 227.94594%, 0)","translate3d(0px, 0px, 0)"]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-bec5c1e0-70a6-40f1-8158-9b8b8f6ba0ed","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-cfc87701-52d9-4d8d-a1a4-b682bd2207a8","keyframes":{"transform":["translate3d(0px, -245.74314%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] Read More

It offers benefits of both: feature selection and preventing overfitting.

[{"selector":"#anim-0c572ccc-fe18-41fc-8c21-f4263b224036","keyframes":{"opacity":[0,1]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-f122c42e-20bf-4e69-bac2-b5b248487d0e","keyframes":{"transform":["translate3d(0px, 210.47254%, 0)","translate3d(0px, 0px, 0)"]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-6a520525-8a59-416b-bc4e-00f85b0a07f2","keyframes":{"opacity":[0,1]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-71b0de27-a4ad-43da-b51e-64253be03234","keyframes":{"transform":["translate3d(0px, 227.94594%, 0)","translate3d(0px, 0px, 0)"]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-7f85f043-c1a1-4347-9a21-ac17f5e47a1f","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-19c3a164-0c65-470f-b16c-a836f662c1d5","keyframes":{"transform":["translate3d(0px, -197.58459%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] Read More

It sets input features to zero during training. Prevents overreliance on single features.

[{"selector":"#anim-38b19662-7413-4ff5-b19e-c54169d4f297","keyframes":{"opacity":[0,1]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-b7a3601b-260b-437e-8c1a-839b63364cbd","keyframes":{"transform":["translate3d(0px, 176.06837%, 0)","translate3d(0px, 0px, 0)"]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-09b786e3-a793-4557-8900-850e389c19ac","keyframes":{"opacity":[0,1]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-b67e2184-8e90-47bf-bfd6-a87ac0e156a4","keyframes":{"transform":["translate3d(0px, 227.94594%, 0)","translate3d(0px, 0px, 0)"]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-3bb0a809-5505-4f51-aea4-15d8899641b4","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-14c1c656-bac3-45ab-8bc2-28b825ceb843","keyframes":{"transform":["translate3d(0px, -245.74314%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] Read More

Stop training before overfitting. Prevents model complexity.

[{"selector":"#anim-115737b8-a2f2-4b9a-be03-ec562cd61a4f","keyframes":{"opacity":[0,1]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-ac6f8e14-f59e-4e48-a2bc-735d0b2ecc03","keyframes":{"transform":["translate3d(0px, 194.98414%, 0)","translate3d(0px, 0px, 0)"]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-d8948b81-f0a4-48f5-aa51-18e00f1085ab","keyframes":{"opacity":[0,1]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-b7cb96a6-351d-44c0-a61c-eb33e0fbd0e8","keyframes":{"transform":["translate3d(0px, 227.94594%, 0)","translate3d(0px, 0px, 0)"]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-4d559861-2581-4b11-b388-2b059f0f9fdb","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-5d1be067-33d1-47ea-aba9-6ea6d01a10ae","keyframes":{"transform":["translate3d(0px, -245.74314%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] Read More

Understand different techniques to build better models.

[{"selector":"#anim-3185d4d7-fedd-485a-913d-a1ee8a972054","keyframes":{"opacity":[0,1]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-29ea315c-cf01-48bc-b846-f5baad188073","keyframes":{"transform":["translate3d(0px, 229.73393%, 0)","translate3d(0px, 0px, 0)"]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-990bdd98-50be-47c3-b1b7-739d2098fc0f","keyframes":{"opacity":[0,1]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-8d8e50aa-e393-411f-bc8c-6f8683ed7f1d","keyframes":{"transform":["translate3d(0px, 227.94594%, 0)","translate3d(0px, 0px, 0)"]},"delay":500,"duration":900,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-44cd86f2-abec-4dc7-a9e8-03183750c074","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-aa05f052-f8ea-4111-a2ab-509b0d898ecc","keyframes":{"transform":["translate3d(0px, -291.42059%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] Read More

NumPy is the go-to library in Python for scientific computing. It supercharges your ability to work with numbers!