- When to do studentization?
- Matlab's comparison of 2 different accuracy metrics for SVM yield huge differences
- R's mvrnorm returning seemingly biased samples
- “Significant” contrasts of random effect intercept levels
- Directly pass from transform feedback buffer to fragment shader
- How to stop the RotateAround function to rotate after Keypress in Unity
- How to customize migs payment gateway page?
- make redirection faster to www-ed and slashed variant
- How to get Steam to recognize existing game files for Arma
- What are minigame ratings used for?
- Where are my Screenshots?
- How do I make an effective Minecraft SMP trap?
- How is the anonymous edit count from a certain location determined?
- Search button lost it's style in stackexchange.com - Newsletters tab
- Drawing/Sketching Tool for elaborating questions
- Sitecore Custom Indexer
- System XML exception for rendering after upgrade
- What’s the difference between sharding and pruning?
- Getting complete state of an smart contract
- MetaMask Gas Limit Problem on Ropsten
Right way to Fine Tune - Train a fully connected layer as a separate step
I'm using Fine Tuning with caffenet and it works really well but then I read this in Keras blog entry on Fine Tuning (They use a trained VGG16 model):
"in order to perform fine-tuning, all layers should start with properly trained weights:
for instance you should not slap a randomly initialized fully-connected network on top of a pre-trained convolutional base.
This is because the large gradient updates triggered by the randomly initialized weights would wreck the learned weights in the convolutional base.
In our case this is why we first train the top-level classifier, and only then start fine-tuning convolutional weights alongside it."
So as a separate step in Fine tuning they save the output of the last layer before the fully connected layer (the "bottleneck features") and then they train a "small fully-connected model" on those features and only then they put the newly trained fully connected layer on top of the whole net and train the "last convolutional block".