- Largest Prime Exponents
- Java String Reverse Algorithm which is the most effcient approch?
- Forming as many pairs as possible with the same sum
- Count of Smaller Numbers After Self
- Tonelli-Shanks algorithm in Python
- Use JQuery to set dynamically the value of an hidden field
- Sort stack in ascending order
- R shiny app to visualize salary data
- Find spanning tree with maximum edges with the same weight
- Converting Array of Floats to UINT8 (`char`) or UINT16 (`unsigned short`) Using SSE4
- Comparator Implementation
- Was Olly convicted for murder or for treason against Jon Snow?
- Deceased attending his own funeral banquet
- I read a book in the 60s where many people had one mental power. One man had many
- Why do Panserbjørns speak English?
- Author and title of fantasy book with ka spirit, energy binder general, orphan artist, barrier wall of shelter, and warring tribes
- If Aman was “taken away” at the fall of Numenor, where do the Elves (and others) sail to at the end of Lord of the Rings?
- Why didn't anyone else hear the Basilisk hiss?
- Can it be true that Panserbjørnes don't have daemons and have no afterlife?
- What is the title of the Wizard of Oz sequel?
Right way to Fine Tune - Train a fully connected layer as a separate step
I'm using Fine Tuning with caffenet and it works really well but then I read this in Keras blog entry on Fine Tuning (They use a trained VGG16 model):
"in order to perform fine-tuning, all layers should start with properly trained weights:
for instance you should not slap a randomly initialized fully-connected network on top of a pre-trained convolutional base.
This is because the large gradient updates triggered by the randomly initialized weights would wreck the learned weights in the convolutional base.
In our case this is why we first train the top-level classifier, and only then start fine-tuning convolutional weights alongside it."
So as a separate step in Fine tuning they save the output of the last layer before the fully connected layer (the "bottleneck features") and then they train a "small fully-connected model" on those features and only then they put the newly trained fully connected layer on top of the whole net and train the "last convolutional block".