Is 100+ Visitors A Day Good Enough For List building?

I have a website that has 100+ visitor according to awstat in cpanel(are they accurate?). Is it worth the effort in building a list? My site is 6th position and with better SEO i hope to gain higher rankings. My traffic also comes from youtube. Do you guys think its a good idea to slowly build my list? Im planning to use aweber $1 trial sign up.

you could use mail-chimp for free

Note: they only allow up to 2000 subscribers on a free account.