Disabling the acceleration profile helps:
xinput --set-prop 'TPPS/2 IBM TrackPoint' 'libinput Accel Profile Enabled' 0, 1
Then you might want to increase your pointer's speed:
xinput --set-prop 'TPPS/2 IBM TrackPoint' 'libinput Accel Speed' 1
Since the next startup the acceleration profile will be enabled again by default you can create a script that includes the above commands and is run automatically after restart.
ref: https://askubuntu.com/questions/1029119/erratic-behaviour-of-trackpoint-on-thinkpad-t420s-ubuntu-18-04