Readers may be familiar with the thought that either or both the birth narratives of Matthew and Luke are creations of the authors with little to do with the facts of the matter (other than that Jesus Christ was born to a woman, as all humans are). It is not hard to imagine each author adding some colour to what they inherited from Mark (who begins the story of Jesus at his baptism). Eventually an eager public wants to know when and how their hero was brought up and so, in their own distinctive ways, Matthew and Luke supply the missing narrative (according to this hypothesis).
The invention hypothesis
Matthew takes the route of imagining Joseph's perspective and what a colourful perspective it is, with dreams in which God speaks, visits from exotic strangers bearing expensive gifts, and an exciting escape to a foreign land from marauding Herod. Matthew's "epiphany" has a (literally) "epic" quality!
Luke takes another route. The alternative to a Josephine perspective is a Marian perspective which Luke takes up with enthusiasm and he includes a bonus-for-his-readers, a second birth narrative, that of John the Baptist (who, we might recall, appears as a significant character just before Jesus makes his appearance in Mark's narrative). Luke has no dreams, no escapes and no sojourn in Egypt, but there are visitations by angels, theologically suggestive songs by leading characters (Zechariah, Mary, Simeon) and a bunch of bewildered shepherds turning up in the middle of the night. There is also an historically awkward presentation of Jesus in the Temple (awkward because it is not at all clear that it was what the law prescribed).
Oh, and to make the inventions of Matthew and Luke more plausible as an explanation of the origin of the narratives, there are the very inconvenient facts that Matthew's genealogy of Jesus does not tally with Luke's, that Luke may be simply wrong about the census under Quirinius (as previously posted here), and the geographical arc of each narrative reverses the other. Matthew's Jesus is born in Bethlehem to (it would appear) Bethlehem citizens but they choose to live in Nazareth after the return from Egypt, whereas Luke's Jesus is born to Mary who with her husband Joseph travel from Nazareth to Bethlehem for the birth and then return to Nazareth. Luke's "epiphany" involves no exotic wise men, just local congregants of the Temple, Simeon and Anna, and his "epic" journey is pretty straightforward and short, Bethlehem to Nazareth. (By contrast, John's Gospel's Prologue's "epic" journey begins before time and its "epiphany" is no less than the revelation that the light which enlightens everything has come into the world.)
Put like that, why would we have any regard at all for either Matthew or Luke's accounts of the birth of Jesus as historical narratives? Why not treat them as sheer fiction relative to the actual but unknown historical facts of Jesus' birth and as narratival theology on a par with the explicitly theological prologue of John's Gospel? And, of course, if we head towards the conclusion that the narratives are not historical, why not cast doubt on the miraculous conception of Jesus?
But let's not leap to an unwarranted conclusion
There are several reasons not to jump towards such a conclusion, at least not to jump to such a conclusion as the complete evaluation of the whole of each narrative.
First, our doubts largely arise because of the nature of the accounts themselves and not because we have an alternative set of facts with which to judge them. It is not as though there is a birth certificate for Jesus which states he was born in Hebron, nor a record of shepherd movements on the night when he was born which proves that all shepherds in the vicinity of his birthplace were sound asleep from dusk till dawn. Neither are there immigration records from Herod's kingdom which tell us the only Magi visiting Palestine in those years arrived from the North in 9 BC, five years too early.
True, there is considerable doubt as to whether Luke has got his facts straight about (a) a census being taken at the likely time of Jesus' birth and (b) such a census requiring people moving to their hometown, but that observation in itself does not rule out the possibility that Jesus was born in Bethlehem, that Mary and Joseph did travel there (for some reason) in time for his birth. It is also true that there is an absence of confirmatory external historical records, notably for something as striking as a massacre of infants in and around Bethlehem, late in the reign of Herod the Great's rule, but absence of confirmation is not confirmation of absence. That is, with the exception of well-known difficulties squaring Luke's account of a census under Quirinius with what we know about Roman censuses in that era, we have no alternative proven historical facts which deny the historicity of either Matthew or Luke's birth narratives. In other words, if the accounts cannot be proven to be historical, neither can they be proven to be unhistorical.
That leads, secondly, to the possibility that there is an interweaving of history and theological interpretation of history going on in these stories, perhaps with more theological interpretation than history. Some interesting aspects of the two birth narratives are stubbornly common to both accounts. Jesus' mother is Mary, his step-father is Joseph. His conception is miraculous. His birth is in Bethlehem. His upbringing is in Nazareth. By themselves these observations do not constitute assured historical facts, but they do raise the possibility that they are facts which Matthew and Luke agree on because they are facts and they cannot be theologised into something else. (One of the oddest features of modern gospel scholarship is that the hypothesized source document Q - defined as the common source for non-Markan material in Matthew and Luke - does not include these common non-Markan aspects of their respective birth narratives).
Then, thirdly, we should consider the intrinsic possibility that a distinctive, indeed unique person such as Jesus Christ had unusual things happening in and around his conception, birth and infancy, even if they look like some kind of convenient theological fiction (written, e.g., as a presumed fulfilment of ancient prophecies, a particular possibility for Matthew's Gospel, or as a rewritten set of stories and songs from Israel's scriptures, a particular possibility for Luke's Gospel).
Moreover, all parents treasure in their hearts the circumstances of their children's coming into the world and early years. It would be historically unusual if Joseph and Mary had no memories of Mary discovering her pregnancy, carrying Jesus to term, his birth and his infancy and even more unusual if they never shared those memories with others and if close family members themselves had no memories of baby Jesus. Thus, why should Matthew not have had access to the Josephine version of events? Although Joseph is largely absent in the gospels beyond the birth and infancy narratives (and thus presumed to have died in Jesus' youth), there is nothing intrinsically improbable about his siblings and cousins, to say nothing of Jesus' own (perhaps older) siblings, having heard from him about Jesus' conception and birth, and told and retold what they heard so that Matthew drills into that particular rich ore of Jesus' personal history. Ditto, Luke drawing on a Marian ore of treasured memories (incidentally, conceivably from Mary herself, if Luke began his gospel investigations in (say) the late 40s and early 50s AD). In other words, there is potential complementarity rather than contradiction between Matthew and Luke's two approaches to telling the birth narrative of Jesus with important common details between the two narratives.
A fourth observation to make is that there is nothing intrinsically improbable within the historiography of Israel (as we understand it from the Old Testament) of a Joseph being communicated to by God through dreams, of angels making announcements to shepherds and of wise men interpreting unusual signs among the stars and the planets. (On the possibility that the star was a particular planetary appearance, see this). Indeed, to this day, the Middle East remains - in my personal experience - a region of the world where many people testify to experiences of divine communication through dreams and angels. Nor, for that matter, is there anything improbable in a Herod feeling threatened by news of a potential rival king and exercising subsequent tyrannical measures to ensure the rival never made it to adulthood. Again, observing these things does not conjure up facts where none actually exist, but such observations should caution those of us with mindsets tempted in a post Enlightenment Western manner to dismiss aspects of the narratives which do not fit the historiography we have developed.
To cut to a conclusion: yes, there are a significant number of reasons to wonder very loudly what is history and what is not in the birth narratives, and indeed to even wonder whether there is any sound history beyond the mere fact that a baby was born who became Jesus of Nazareth. But there are also significant reasons for not moving from wondering about these things to determining that we "know" there is little or no history involved in the narratives. We do not have facts at hand to make such determination. In the end, the narratives are the history of Jesus' birth and infancy we have as God's people. There is no other history available to us.
Postscript: a much better post (regarding detailed examination of the text in relation to the question of historicity) is here, by Ian Paul.