You would need do compile your own copy of R and increase the stack
size,
size,
That depends on your unstated OS. Uwe gave you an answer for Windows.
On many other OSes, the stack size is set up by the OS when the
process is started, and your OS documentation will tell you how to
increase it.
Xiaobo Gu