Q: Sparql query returned value integer, cant filter it to get the int Although the query returns the country names without filter the population, while i use the filter to take the integer returns nothing.
PREFIX  dbo:  <http://dbpedia.org/ontology/>

SELECT DISTINCT  str(?country_label) ?population
WHERE
  { 
?country  dct:subject  <http://dbpedia.org/resource/Category:Countries_in_Europe>.
?country rdfs:label ?country_label.
?country dbo:populationTotal ?population.
FILTER langMatches( lang(?country_label), "EN" )
FILTER datatype((?population) = xsd:integer ) 

 }


A: I'm not sure what you're trying to achieve with this FILTER condition, but the reason your query returns zero results is that none of the population values are of datatype xsd:integer. 
As an aside there is something else wrong in your query: using a function like str() in your SELECT clause without binding the result of that function to a new variable (using as ?new_var) is not legal SPARQL. I know the DBPedia endpoint allows it, but you should be aware it's likely to give you errors if you ever try it on a different SPARQL engine.
If you run the query without the datatype filter:
PREFIX  dbo:  <http://dbpedia.org/ontology/>
SELECT DISTINCT  (str(?country_label) as ?cl) ?population
WHERE
  { 
?country  dct:subject  <http://dbpedia.org/resource/Category:Countries_in_Europe>.
?country rdfs:label ?country_label.
?country dbo:populationTotal ?population.
FILTER langMatches( lang(?country_label), "EN" )

}

You will see that the result contains only values of datatype xsd:nonNegativeInteger. That's a different datatype, and although it is of course true that every non negative integer is also an integer, datatype inference  of this form is not something that most SPARQL endpoint support.
Also: in your original filter condition on population, you have the parentheses placed wrong. If you modify your query as follows:
PREFIX  dbo:  <http://dbpedia.org/ontology/>
SELECT DISTINCT  (str(?country_label) as ?cl) ?population
WHERE
  { 
?country  dct:subject  <http://dbpedia.org/resource/Category:Countries_in_Europe>.
?country rdfs:label ?country_label.
?country dbo:populationTotal ?population.
FILTER langMatches( lang(?country_label), "EN" )
FILTER ( datatype(?population) = xsd:nonNegativeInteger ) 

 }

You will get back all values that are of that datatype - though, as said, I'm not sure what you're achieving here, as this is the same result you'd get without the datatype-check, as far as I can tell. 
If your goal is to get the number back, but represented as an xsd:integer instead of as a xsd:nonNegativeInteger, you can of course cast it, instead:
PREFIX  dbo:  <http://dbpedia.org/ontology/>
SELECT DISTINCT  (str(?country_label) as ?cl) (xsd:integer(?population) as ?pop)
WHERE
  { 
?country  dct:subject  <http://dbpedia.org/resource/Category:Countries_in_Europe>.
?country rdfs:label ?country_label.
?country dbo:populationTotal ?population.
FILTER langMatches( lang(?country_label), "EN" )
 }

