Possible bug affecting mpz_popcount() in GMP 5.0.1 on Intel Core2

[Roberto Bagnara]

Can someone please try to reproduce the following?

$ cat p.c
#include <gmp.h>

int main() {
   mpz_t x;
   mpz_init(x);
   mpz_set_ui(x, 3);
   return mpz_popcount(x);
}
$ gcc -W -Wall -g p.c -L/usr/local/lib64 -lgmp
$ valgrind --db-attach=yes ./a.out
==20177== Memcheck, a memory error detector
==20177== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al.
==20177== Using Valgrind-3.6.0.SVN-Debian and LibVEX; rerun with -h for copyright info
==20177== Command: ./a.out
==20177==
==20177== Invalid read of size 8
==20177==    at 0x4E5F8EA: __gmpn_popcount (in /usr/local/lib64/libgmp.so.10.0.1)
==20177==    by 0x50BBC4C: (below main) (libc-start.c:226)
==20177==  Address 0x5420048 is 0 bytes after a block of size 8 alloc'd
==20177==    at 0x4C274A8: malloc (vg_replace_malloc.c:236)
==20177==    by 0x4E39638: __gmp_default_allocate (in /usr/local/lib64/libgmp.so.10.0.1)
==20177==    by 0x4E48C27: __gmpz_init (in /usr/local/lib64/libgmp.so.10.0.1)
==20177==    by 0x4006F7: main (p.c:5)
==20177==
==20177==
==20177== ---- Attach to debugger ? --- [Return/N/n/Y/y/C/c] ---- y
==20177== starting debugger with cmd: /usr/bin/gdb -nw /proc/20185/fd/1014 20185
GNU gdb (GDB) 7.1-ubuntu
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /proc/20185/fd/1014...done.
Attaching to program: /proc/20185/fd/1014, process 20185
Reading symbols from /usr/lib/valgrind/vgpreload_core-amd64-linux.so...done.
Loaded symbols for /usr/lib/valgrind/vgpreload_core-amd64-linux.so
Reading symbols from /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so...done.
Loaded symbols for /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so
Reading symbols from /usr/local/lib64/libgmp.so.10...(no debugging symbols found)...done.
Loaded symbols for /usr/local/lib64/libgmp.so.10
Reading symbols from /lib/libc.so.6...Reading symbols from /usr/lib/debug/lib/libc-2.11.1.so...done.
done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib/debug/lib/ld-2.11.1.so...done.
done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
0x0000000004e5f8ea in __gmpn_popcount () from /usr/local/lib64/libgmp.so.10
(gdb) frame 1
#1  0x0000000000400715 in main () at p.c:7
7	  return mpz_popcount(x);
(gdb) print x
$1 = {{_mp_alloc = 1, _mp_size = 1, _mp_d = 0x5420040}}
(gdb) print sizeof(mp_limb_t)
$2 = 8
(gdb) frame 0
#0  0x0000000004e5f8ea in __gmpn_popcount () from /usr/local/lib64/libgmp.so.10
(gdb) x/i $pc
=> 0x4e5f8ea <__gmpn_popcount+42>:	movdqa (%rdi),%xmm0
(gdb) print/x $rdi
$3 = 0x5420040
(gdb)

In words: at address 0x5420040, 16 bytes are read from a buffer of 8 bytes.

I also have

$ /usr/local/distrib/gmp-5.0.1/config.guess
core2-unknown-linux-gnu
$ cat /proc/cpuinfo
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 23
model name	: Intel(R) Core(TM)2 Quad CPU    Q9400  @ 2.66GHz
stepping	: 10
cpu MHz		: 2003.000
cache size	: 3072 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm tpr_shadow vnmi flexpriority
bogomips	: 5327.22
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
...

I believe the code in question comes from
mpn/x86/pentium4/sse2/popcount.asm.
The problem does not show up on another machines I have
that mount AMD processors (using code in mpn/popcount.asm).

If someone can confirm the above observations I will submit
a bug report.
Cheers,

    Roberto

-- 
Prof. Roberto Bagnara                     CEO & CTO
Applied Formal Methods Laboratory         BUGSENG srl
Department of Mathematics                 Parco Area delle Scienze 53/A
University of Parma, Italy                I-43124 Parma, Italy
http://www.cs.unipr.it/~bagnara/          http://bugseng.com/
mailto:bagnara at cs.unipr.it                mailto:roberto.bagnara at bugseng.com