  The HIFI data processing pipeline was developed to systematically process
diagnostic, calibration and astronomical observations taken with the HIFI
science instrumentas part of the Herschel mission. The HIFI pipeline processed
data from all HIFI observing modes within the Herschel automated processing
environment, as well as, within an interactive environment. A common software
framework was developed to best support the use cases required by the
instrument teams and by the general astronomers. The HIFI pipeline was built on
top of that and was designed with a high degree of modularity. This modular
design provided the necessary flexibility and extensibility to deal with the
complexity of batch-processing eighteen different observing modes, to support
the astronomers in the interactive analysis and to cope with adjustments
necessary to improve the pipeline and the quality of the end-products. This
approach to the software development and data processing effort was arrived at
by coalescing the lessons learned from similar research based projects with the
understanding that a degree of foresight was required given the overall length
of the project. In this article, both the successes and challenges of the HIFI
software development process are presented. To support future similar projects
and retain experience gained lessons learned are extracted.
