How to manage memory with GMP’s mpz_fac_ui and Visual C/C++ 6.0?

[Bruce M. Axtens]

I have the following C code which calls the factorial function in GMP 4.1.x.

#define Z(x) mpz_t (x); mpz_init( (x) );
#define VALLOC(nBytes) VirtualAlloc( NULL, (nBytes), MEM_COMMIT, 
PAGE_EXECUTE_READWRITE );
#define VFREE(aAddr) VirtualFree( (aAddr), 0, MEM_RELEASE ); 

BSTR __stdcall IBIGFACT( unsigned long p1 ) {
        USES_CONVERSION;
        Z(rop);
        mpz_fac_ui( rop, p1 );
        size_t sib = mpz_sizeinbase( rop, 10 );
        char * buff = (char *) VALLOC( sib + 2 );
        mpz_get_str(buff, 10, rop);
        BSTR bResult = _com_util::ConvertStringToBSTR( buff );
        VFREE( buff );
        return bResult;
}

Annoyingly, I can't use this to calculate 234234! because I run out of 
stack space trying to calculate 112094!.

When I try to calculate 234234!, sib ends up containing 1156032. 
VirtualAlloc therefore tries to allocate 1156034 bytes, which it does. 
It's when the mpz_get_str function executes that the stack error appears.

Interestingly, Frink uses GMP and calculates the factorial of 234234 
flawlessly and generates about 1.1MB of digits. Why won't mine?

Now as it turns out, DLLs can't set their own stack size -- that's 
something done on the main app. But this DLL is being used by a COM DLL 
and the COM DLL by Excel. I have my doubts that upping Excel's stack 
size is going to help much.

Moving away from VirtualAlloc to gmp's own allocation mechanisms doesn't 
help either.

BSTR __stdcall IBIGFACT( unsigned long p1 ) {
    USES_CONVERSION;
    char * buff;
    Z(rop);
    mpz_fac_ui( rop, p1 );
    size_t sib = mpz_sizeinbase( rop, 10 );
    buff = mpz_get_str(NULL, 10, rop);
    BSTR bResult = _com_util::ConvertStringToBSTR( buff );
    return bResult;
}