Which memory is used for external process run from java - java heap space or OS memory?

If I run the following line

final String[] command = new String[]{ffmpeg -y -i /home/user/video.mov -ss 0 -t 20 -vcodec libx264 -vpre slow -crf 18 -f flv -bf 0 -g 10 -vsync 1 -r 30 -an -threads 0 -s 1920x1080 /home/user/video0.flv};
 
 final Process process = Runtime.getRuntime().exec(command, null, null);
 

It will start ffmpeg and convert first 20 seconds of a video.mov to video.flv. But sometimes with high resolution videos, OutOfMemory is thrown. Obviously ffmpeg takes too much memory.

My question is the following - are external processes started from java taking memory from java heap space or from OS memory?

Knowing this, I will know how to adjust -Xms and -Xmx parameters. If external processes take from OS memory, I will leave -Xms and -Xmx with low values(leaving OS with more free memory). Otherwise, I will set -Xms and -Xmx to high values, giving the Java process more memory.

---