Magnetic resonance imaging (MRI) in the ultra-low field (ULF) regime (<< 1 T) is a promising modality for anatomical imaging with multiple advantages over conventional several-Tesla MRI, such as low cost and low weight. In particular, ULF MRI allows combination of magneto-encephalography (MEG) and MRI in a single instrument which is uniquely capable of linking the sources of biomagnetic brain activity to the specific anatomical brain structure with both excellent temporal and spatial resolution. The combination of the two modalities will enhance understanding of human brain function, aid in diagnosis and treatment of multiple brain disorders such as the epileptic focus, and improve neurosurgical planning. Previously, the MEG-MRI combination was realized only using multiple cryogenic superconducting quantum interference device (SQUID) sensors. However, the demand for cryo-cooling and a shielded room is a major drawback. Recently, we started a project for building a more practical device by replacing SQUIDs with a novel type of atomic magnetometers (AMs). Based on lasers and alkali-metal vapor cells, AMs are currently the most sensitive cryogen-free magnetic sensors. Here we will present our approach in detail and report our recent progress.