I am just wondering should I use the proxy at the time of FB new account creation or should I use it all the time even when logged in from that FB account created using that proxy?
Will logging in from the same IP as mine without proxy after FB new account creation from a proxy affect my account?

Yes it may, however this is all speculation as no one knows facebooks exact banning algorithm.

However. If you saw an account created with an ip from Chicago and then saw that account logged in 10 mins later in new york it may look sketchy.
I think one or two ip changes is fine but try to log onto the account with the same ip address everytime.

-Good Luck