|How the sausage is made|
Suburban sprawl is the inevitable result of community planning that dictates building extensive tracts of low-density housing and a fairly strict segregation of development into commercial, residential, and industrial zones of various sorts. Originally a symbol of postwar prosperity (as well as white flight out of the cities), by the 1970s much suburban development had come to be seen in environmentalist and leftist circles as just as much blight as the urban poverty left behind.
This sprawl was encouraged by the twin post-war "opportunities" of cheap steel (with which to build cars) and cheap oil. It built on a quasi-rural lifestyle myth very common with Americans -- the desire to live on and in "the land", to have a nice leafy green backyard, and to have a nice chunk of property on which to own one's "castle".
The extensive development of suburban sprawl has been blamed for a lot of social ills, the greatest being the loss of the "walkable" community, confining residents to their cars for most activity and therefore reducing available opportunities for exercise, and the decay of downtown areas in favor of shopping malls and big box retail stores, often raising the barrier of entry for small food service and retail businesses in many areas. Public transportation is often inadequate or, more usually, entirely absent. Additionally, many such developments were built on what had formerly been farmland or open space. The wildfires of the last decade have particularly affected houses built on former forest land.
Proposals have been floated for solving some of the problems of suburban sprawl, but apart from a few recent attempts to restore some kind of mixed-use zoning to most suburban areas, most of them involve tweaking traffic flows and attempting to restore walkability. More extreme examples — Paolo Solieri's concept of the arcology, for example — have been floated, but are seldom taken seriously due to their impracticality and often-strained, or even false, assumptions about human nature. Vertical farming in urban high rises or farmscrapers is another idea, however certain crops such as wheat maize and rice are difficult to grow in these conditions. 
The issue continues unresolved, especially in light of the late-2007/early-2008 mortgage crisis that left vast tracts of houses either foreclosed or unoccupied. Leigh Gallagher, author of “The End of the Suburbs: Where the American Dream Is Moving" estimates that by 2025 the majority of homes in the suburbs will have no children. 
Why people move to the suburbs
While urban sprawl is most notable in the US, the US is far from the only country where suburbs of this type exist. In general the factors that make people move to the suburbs can be broadly grouped into economic, lifestyle and political factors.
Land costs less in the suburbs, which means that a house of the same size will be more affordable in a suburb than closer to the city. Hence the phrase "drive till you qualify" - that is, the prospective renter or buyer should start at their place of employment, downtown or whichever relevant point and drive as far out as necessary until a place is reached where rents or house prices are within the means of the prospective tenant. This of course ignores the costs of commuting, which was a safe bet in the US when gas was fifty cents a gallon, but it makes less sense in places where commuting is more expensive.
Simply put, some "conveniences" are simply unaffordable in urban areas. Among others the space needed for a private pool (as opposed to - shudder - going to the public pool with other people), two cars in the driveway and a meticulously maintained lawn that is never used for anything useful is simply unaffordable in the city. On the other hand, a tract of land where there is only residential development for miles on end means that for such simple acts as buying a bottle of milk or an apple, people will have to get in a car and drive. In a dense mixed use development, people can just walk across the corner to the deli or get their lazy ass on a bike and drive to that new Vietnamese place a mile or two away.
A factor that is often only discussed in hushed tones of voice but that nonetheless greatly influences the decision to live in the burbs is the desire to have "good schools" for children and to get away from "the bad part of town". There is a reason why most suburbs are more than 90% one ethnic group even today. Similarly, suburbs that are incorporated into the neighboring city in all ways except the political often resist annexation, because "white taxes" would then be "wasted" on public libraries in poor neighborhoods or to build a public pool. And who wants that?
One strategy that can minimize this driver of suburban sprawl is to merge city and county governments, which ends the free rider problem where suburbanites living outside city limits use city services without having to pay city taxes.
Why people hate the suburbs
In general millennials in particular hate the suburbs because there is literally nothing you can do in the average suburban neighborhood that requires leaving the house if you don't have access to a car. As gas prices have risen, congestion refuses to go away even with seventeen-lane freeways and millennials tend to dislike driving, suburbs have increasingly lost their appeal. And while the past two generations have tended to move out to the suburbs once children were on the horizon or shortly after the birth of the first one, the millennial generation that mostly grew up in the suburbs wants to spare their children the same pain they experienced growing up and increasingly raise children in the city, despite all the crime and bad schools and brown people and despite the fact that children may need to learn how to ride the metro before they turn thirty if they grow up in the city.
The arcology is essentially a concept for hyper-urban living, an enormous super-apartment development containing all, or nearly all, the functions of a fully-developed city in one or a very few buildings. It was coined by Italian architect Paolo Soleri (1919–2013). The word is a portmanteau of "architecture" and "ecology," and was created as a reaction to suburban sprawl, the idea being that a high-density, small-footprint development would have less impact on the surrounding environment than car-based suburbs. Solieri's proof-of-concept, the town of Arcosanti, Arizona, has so far been more of an educational endeavor than an actual working arcology.
Essentially an arcology would fold into one structure most or all of the functions of a typical town, including residential and commercial space, light industry, schools, and perhaps even agriculture (using greenhouses and, perhaps, hydroponics). (There are skyscrapers even now, such as the John Hancock Tower in Chicago, that in combining residential and commercial space in one structure, approach the ideal of an arcology, but are not quite self-sufficient enough to qualify.)
- "Vertical Farming Explained: How Cities Could be Food producers of the Future", Nicola Davis, The Guardian February 6, 2014.
- "Is Suburban Sprawl on Its Way Back?" Shalia Dewan, New York Times Sept. 14, 2013.