Here is short script doing that, using xargs capabilities:

#!/bin/bash
SOURCE_DIR="/sourcedirectory"
TARGET_DIR="targetuser@targethost:/targetdirectory"
THREADS="8"
cd ${SOURCE_DIR}; find . -type f | xargs -n1 -P$THREADS -I% scp % ${TARGET_DIR}/%