MPILIB: Fix comparison of negative MPIs
authorRasmus Villemoes <linux@rasmusvillemoes.dk>
Wed, 14 Jan 2015 16:10:12 +0000 (16:10 +0000)
committerDavid Howells <dhowells@redhat.com>
Wed, 14 Jan 2015 16:10:12 +0000 (16:10 +0000)
If u and v both represent negative integers and their limb counts
happen to differ, mpi_cmp will always return a positive value - this
is obviously bogus. u is smaller than v if and only if it is larger in
absolute value.

Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Signed-off-by: David Howells <dhowells@redhat.com>
Acked-by: Dmitry Kasatkin <dmitry.kasatkin@gmail.com>
lib/mpi/mpi-cmp.c

index 8ed36b8cbe02b32b096941cdb196b88b67d66d98..d25e9e96c310fc8aeeaff05213ac74b9f6280a96 100644 (file)
@@ -57,7 +57,7 @@ int mpi_cmp(MPI u, MPI v)
        if (usize != vsize && !u->sign && !v->sign)
                return usize - vsize;
        if (usize != vsize && u->sign && v->sign)
-               return vsize + usize;
+               return vsize - usize;
        if (!usize)
                return 0;
        cmp = mpihelp_cmp(u->d, v->d, usize);