Unless cared for on a regular basis, the fruit from apple trees will often get smaller as each year passes. This is often accompanied by a general paling of the foliage, but not always noticeable, as many apple tree varieties have pale green foliage anyway. However if the foliage is unevenly coloured with lighter areas between the veins, then it is almost certainly 'starvation'.
The cause of apples getting smaller fruits each year is usually as a result of lack of nutrient - Nitrogen in particular. But the can be other deficiencies. Whilst it is easy to do a localised soil nutrient test, it may not be accurate for the soil region down deeper in the soil where the Apple tree is actually feeding from.
Maybe there is nutrient in the soil or container, but it is not accessible to the tree. Nutrient 'lock-out' is more common than is realised.
It may be a year or so until the apples get back to their proper size, but at least it will all stop your apples from getting smaller each year.
I have a commercial grower friend who feeds his trees Nitrogen every autumn. This goes against all advice, but he maintains that the trees take up the nitrogen late autumn, and are ready to utilise it early spring. It certainly works for him, so it may be worth a trial on a tree or so. I would not recommend it as normal practice each autumn.