sg engine: errno return value fixes

sg engine: errno return value fixes

It was a little all-over-the-place, unify it - return -1 for error,
let the caller look at errno for the specific error.

Signed-off-by: Jens Axboe <jens.axboe@oracle.com>