We do know that We have to wash our hands in the downwards direction, but many times due to human eror, we wipe a little in upwards direction as well without knowing. This just happens by accident. Does it still invalidate the wudhu and we have to start again or is it fine?

That’s fine on problem