For much of our history, America was predominantly an agrarian society. Law makers took into account that November was perhaps the most convenient month because the fall harvest was in, and the majority of the nation's weather was still mild enough to permit travel over unimproved roads.
Since most residents of rural America had to travel a significant distance to the county seat in order to vote, Monday was not considered reasonable since many people would need to begin travel on Sunday. This would have conflicted with church services and Sunday worship.
Why the first Tuesday after the first Monday? Lawmakers wanted to prevent election day from falling on the first of November for two reasons. First, November 1st is "All Saints Day", a holy Day of Obligation for Roman Catholics. Second, most merchants were in the habit of doing their books from the preceding month on the 1st. Apparently, Congress was worried that the economic success or failure of the previous month might prove an undue influence on the vote!