Clear ->file_map after free()