I'm not sure that I have enough information to successfully answer your question, as I do not have access to the notes and therefore the derivation. That being said it might be helpful if I say something about the self-financing property.

A portfolio is self-financing if and only if there is no external capital being included into the portfolio value through time. This means that at all times t where the portfolio gets rebalanced:

$$\underline{\theta}_t \bullet \underline{S}_t = \underline{\theta}_{t+dt} \bullet \underline{S}_t = V(t)$$ Where theta is a vector of portfolio holdings, S is a vector stock price values and V is the portfolio value. The bullet operator here is the dot product or "sum-product" of the vectors.

Note that theta - the portfolio holdings - are then fixed for the subsequent period over which the investment gain occurs. What is means is that the gain in the portfolio for times t to time t+dt is going to be entirely dependent on the change is S as below $$V(t + dt) - V(t) = \underline{\theta}_{t+dt} \bullet \underline{S}_{t + dt} - \underline{\theta}_{t+dt} \bullet \underline{S}_{t} = \underline{\theta}_{t+dt} \bullet d\underline{S_t}$$

The only random thing in the above term is the S.

This is why the notes say "the risk free portfolio is not self financing because the pure investment gain derived is not equal to the instantaneous change in the value of the portfolio".

The change in value, the LHS of the above equation, is equal to the pure investment gain from the buy and hold strategy of the theta portfolio over the period.

If you'd like to write down your derivation and the contradiction in question more fully I can attempt to answer your question more specifically.

Could I see how you derived V(t+dt, St+dt) - V(t, St)?