After losing the 2012 presidential election on Nov. 6 to President Barack Obama, Mitt Romney is spending quality time with his family.

The former presidential hopeful enjoyed a low-key dinner out without secret service at Lumiere restaurant in Newton, Mass. Saturday night, a source tells Us Weekly.

PHOTOS: Stars' political affiliations

Romney, 65, dined with wife Ann, 63, and some of their family members, an eyewitness says. (Mitt and Ann are parents to sons Taggart, Matthew, Joshua, Benjamin and Craig. They also have 18 grandchildren, with one more on the way.)

Though the source tells Us that Romney "looked a little down," his wife was cheering him up and "being affectionate."

PHOTOS: Stars who played the president

During his concession speech, Romney thanked his family for their support.

"I also want to thank Ann, the love of my life. She would have been a wonderful first lady. She has been that and more to me and to our family and to the many people that she has touched with her compassion and her care," the former governor of Massachusetts said. "I thank my sons for their tireless work on behalf of the campaign, and thank their wives and children for taking up the slack as their husbands and dads have spent so many weeks away from home."

PHOTOS: Stars vote in the 2012 election

"I so wish that I had been able to fulfill your hopes to lead the country in a different direction, but the nation chose another leader," Romney concluded. "And so Ann and I join with you to earnestly pray for him and for this great nation."

This article originally appeared on Mitt Romney Enjoys Family Dinner Out With Wife Ann, Grandchildren After Election Loss