The-Last-Lion-Turtle
The-Last-Lion-Turtle t1_j89jm9s wrote
The purpose of a deep network is to approximate complex non linear functions. With relu the network is piecewise linear. Imagine slicing a space with many planes, locally it's flat, but zooming out it has a very complex shape, similar to getting a 3D model out of triangles. Each layer adds an additional linear deformation and a slice to the space.
Read the resnent paper. It's a great explanation for both why depth matters for performance and how it causes issues for training. The solution of residual connections is central to every deep learning architecture after this paper.
The-Last-Lion-Turtle t1_j7j1qxl wrote
Reply to Wouldn’t it be a good idea to bring a more energy efficient language into the ML world to reduce the insane costs a bit?[D] by thedarklord176
Pytorch is written in C++ and CUDA.
Python is really just an interface, with a minimal contribution to the execution time.
The-Last-Lion-Turtle t1_iwbi5jv wrote
It doesn't look like there are any convolutions in that net. Fully connected layers don't work that well.
Resnet or wide resnet would be a better idea.
The-Last-Lion-Turtle t1_isqdzma wrote
Reply to comment by St0nks_Only_Go_Up in Australian research finds cost-effective way to recycle solar panels | Recycling by DrDaleks
That's pretty much what my comment said.
The-Last-Lion-Turtle t1_iskjz7r wrote
Reply to comment by lacergunn in Australian research finds cost-effective way to recycle solar panels | Recycling by DrDaleks
Neat
I assume it's still easier to prevent the pollution than clean it up.
The-Last-Lion-Turtle t1_iskiwrb wrote
Reply to comment by lacergunn in Australian research finds cost-effective way to recycle solar panels | Recycling by DrDaleks
Heavy metals or just typical ones like iron and copper?
The-Last-Lion-Turtle t1_isk06gy wrote
Reply to comment by kmcclry in Australian research finds cost-effective way to recycle solar panels | Recycling by DrDaleks
Recycling is definitely relevant, it's just the primary problem is infrastructure not getting individual people to use a different colored bin.
The-Last-Lion-Turtle t1_isjsov4 wrote
Reply to comment by Hawk---- in Australian research finds cost-effective way to recycle solar panels | Recycling by DrDaleks
China is having an ewaste crisis with heavy metal pollution in their ground water, because their "recycling" is really just getting paid by other countries to dump it somewhere else.
The-Last-Lion-Turtle t1_j8jrena wrote
Reply to [D] Sentient AI Encryption by Prestigious_Tap8633
It could work with a heavy efficiency penalty.
https://en.m.wikipedia.org/wiki/Homomorphic_encryption
Though I don't think gradient descent will select for something like this.
Far more likely is to obfuscate how it works so while it's not encrypted we learn little with the tools we have, and would have an extremely difficult time verifying something is absent.