- Strange object on 737-800 exterior?
- Requirements for complex and high-performance endorsement?
- How come nobody survived Air Moorea 1121, yet other people have survived crashes from altitudes more than 90x higher?
- Why is the Eurofighters nose gear door shorter than the bay?
- What does “v'ha” mean?
- “New TOR Circuit for this Site” changes all tabs IP addreses
- How to completly remove influence of bone to mesh
- Motion Blur at different frames
- Trackpad with Surface Book cannot navigate 3D View
- Tracking scene moves markers from points in 3D view
- Smooth transition between distorted and undistorted result of Movie Distortion node
- How do you access the ruler?
- Object not showing up in full render
- How to say “pay exact amount of money”
- How to use the word 'suitable' properly?
- why the answer in this question is “(A)”?
- What's the difference between drug and medication?
- Which auxiliary verb or copular to refer to uncountable nouns?
- Is there difference between bedding, bedclothes and linen or are consider synonym?
- What would you call someone who sneaks into and stays in places where they have to be signed up, e.g. classes/country clubs
Right way to Fine Tune - Train a fully connected layer as a separate step
I'm using Fine Tuning with caffenet and it works really well but then I read this in Keras blog entry on Fine Tuning (They use a trained VGG16 model):
"in order to perform fine-tuning, all layers should start with properly trained weights:
for instance you should not slap a randomly initialized fully-connected network on top of a pre-trained convolutional base.
This is because the large gradient updates triggered by the randomly initialized weights would wreck the learned weights in the convolutional base.
In our case this is why we first train the top-level classifier, and only then start fine-tuning convolutional weights alongside it."
So as a separate step in Fine tuning they save the output of the last layer before the fully connected layer (the "bottleneck features") and then they train a "small fully-connected model" on those features and only then they put the newly trained fully connected layer on top of the whole net and train the "last convolutional block".