PropertyFor ignores custom image template/view

I have a wierd issue where my custom template for images is not being used when using PropertyFor. I followed the example in developer training to create the ImageFile class with a controller and view, but it seems to be ignored when using PropertyFor with a ContentReference. It does work right when images are inside in a ContentArea property though. I have the view for the ImageFile in the ~/View/ImageFile/ folder as index.cshtml. Here is my implementation, can anyone give me an idea why this would happen?

    public class ImageFileController : PartialContentController<ImageFile>
    {
        private readonly UrlResolver _urlResolver;
        public ImageFileController(UrlResolver urlResolver)
        {
            _urlResolver = urlResolver;
        }
        public override ActionResult Index(ImageFile currentContent)
        {
            var model = new ImageViewModel(currentContent, _urlResolver);
            return PartialView(model);
        }
    }