The automatic reaction to all of these problems seems to be 'a load of topsoil' will do the trick. However, if the basics are tackled, then the load of topsoil is probably not required - not the best course of action, for you will still have the 'poor' soil that needs attention not covering up!
If the soil is compacted by too much use over the years, or builders machinery, or even kids and dogs running around in wet soil conditions, then the soil surface has to be broken up before any other remedial actions. This can be accomplished quite simply by just prising a strong garden fork into the top six or so inches of the compacted soil surface and levering back and forth. This will immediately help with surface drainage and also allow air down into the soil.
Remove all the brick rubble and builders material that you can, then spread a load of manure - Rotted farmyard manure or mushroom compost will do the job. Also your local authority will probably have supplies of treated sewage sludge or maybe amenity refuse that has been composted down. Do not use tree chippings for this, as in the initial stages of decomposing, they will take much goodness out of the soil. Composted bark is ok, but will be expensive. Your soil needs feeding - not just covering up!
If this organic layer is simply left on top of the compacted soil during the autumn and winter months, much of it will naturally break down and be taken down into the soil by various actions of insects and and of course earthworms. Let them do the hard work for you.
The following spring, you should be able to fork a planting area over without too much trouble. Here again you can continue to remove debris out of the soil as and when it comes to light. Just start getting some crops into the soil and continue with general cultivation. this will all help to give you friable, fertile soil to work with. Far better than trying to do it with one back-breaking operation. In one year, you should be able to turn a rubbish heap into a productive vegetable plot. Yes, there will be more work and manure required, but you will be able to see and taste the fruits of your labour quite quickly.
That is the incentive to carry on with the good work of clearing your rubble, sorting out your compacted soil and start gardening with your fertile vegetable plot!