Wife owns a 2008 2 Door Yaris Hatchback - and she LOVES IT - (Last car for her was a F-150 - so big change)
I have a 2008 Passion
Can't compare the two cars fairly after all one does have a REAR seat.....
Personally both are great
I think the Yaris will out last my smart and will cost less to maintain...... I put 252,000 miles on a Corolla and it simply kept going - I gave it away to someone else who now drives the car.... wow......
Yaris - more pick up and peppy; more storage with the rear seats down (actually less than the smart if the seats are up.
Yaris - way more storage areas all over the place - (smart dropped the ball here

)
Yaris - local autopart store has most parts for general maintainence readily available
Smart - much better mpg
Smart - IMO - better looking
Smart - panoramic see through roof
Smart - standard features (ABS, ESP, heated mirrors etc) - if you add all the extras you get standard on a Passion to a Yaris - the difference in cost is substantial - I'd know - I paid for both
Smart - stick (kind of) and auto - so much fun
Having both in my family is good as they both are economical and cheap to buy and insure - the rear seat of the Yaris is handy (often)
People have come up to the Yaris to announce how nice it looks - though noticeably not as much as happens with the smart
We drove the Fit - wife did not like it - oh well...... also looked at Aveo, Esteem, Accent etc - side note - All of these small cars look cheap because they are - so without isolating anyone of them - globally they all look like cheap plastic.... erm.... (not bias) except the smart - inside it certainly doesn't look cheap and empty....
Oh - and for safety - both the Yaris and Smart are supposedly equals in crash testing
As an additional note - I was going to buy 2 Yaris' but held out for my smart and I am personally glad I did - though I didn't need a car urgently as I had 3 other vehicles...
Cheers