A failed state is a recognized state that is incapable of fulfilling the basic requirement of being a state, i.e. does not hold a monopoly on the "legitimate means of coercion." This means that the state's apparatus does not have the legitimacy of its people in matters of punishing criminals or defending borders. This can be for a few different reasons. This is usually due to citizens of the state being too invested in some subgroup within the state, such as tribal affiliation or religious organizations. However, it can also come because the existing state apparatus has completely abdicated the role, or else because the state has lost legitimacy but no one else has been able to put forward a competing claim strong enough to rally the population.
Not every failed state exists in complete anarchy, but there is often some degree of it. Technically, a state in which non-governmental actors (such as tribes or corporate associations) keep the peace while an ineffective central government does nothing would be considered a failed state, but it would be expected for the true holders of the monopoly on legitimate coercion to become the state. The term "failed state" is usually only applied to places where there are NO actors who have legitimacy across the state.
The classic example of a failed state is Somalia, though the current situations in Iraq and Afghanistan would also qualify. However, it should be understood that "failed state" can exist in a kind of spectrum. No one considers North Korea to be a functioning state, but the government still has a modicum of legitimacy (it seems). Zimbabwe is also on the road to being a failed state.