Well, no. What clients care about is what you can offer them. If you're concerned about a meeting place, meet him at his favorite resto-bar, or to his nearby coffee shop.
If you do get questions like "Why do you work at home?" , just tell them that you find the workflow of not having an office...