Conditional Sorting in ElasticSearch

Even though I'm going to code this on EpiServer Find, first I want to solve my problem in ElasticSearch query. I have an object in my index which has a property as 

"SearchPromotion": [
        {"Order$$number": 8,"ItemForSearchPromotion": "109"
        },
        {"Order$$number": 8,"ItemForSearchPromotion": "10"
        }
    ]

What I want to do is really basic, a conditional sorting like this as described in query

"sort": [
        {"SearchPromotion.Order$$number": {"order": "desc","missing": "_last","ignore_unmapped": true,"nested_filter": {"term": {"SearchPromotion.ItemForSearchPromotion": "3"
                    }
                }
            }
        }
    ]

But the problem is, even I send a filter term which is not there like "foo" it returns with value 8 in sort for this particular object, where it supposed to turn -Infinity. Can someone help me here to tell me what am I doing wrong?

Kind regards.