Barthélémy von Haller, software engineer in the ALICE DAQ group
A.M. How long have you been working in ALICE?
B.v.H. I have been working in ALICE almost for four years. Previously I worked at CERN in the IT department as a Marie Curie Fellow. Then I applied for a CERN fellowship and that’s how I got in touch with the DAQ team of ALICE and started working with them on the data quality monitoring software. After the end of my fellowship I moved to EPFL and worked on a totally different project on neuroscience. Finally, a staff position opened here at CERN for which I applied and here we go!
A.M. Which are the main tasks/responsibilities of your post?
B.v.H. As a software engineer, part of my work in ALICE is linked to the development of the Data Quality Monitoring (DQM) framework called AMORE. The basic idea behind AMORE is that we don’t want to wait until the data are fully analyzed and reconstructed to find out whether everything went well. Instead we need to know as fast as possible whether the detectors were calibrated or if a different problem arose. So the data quality monitoring consists of collecting samples of data, analyzing them with detectors specific algorithms and publishing results that will be looked at by either the experts or the DQM shifter. For the latter we have a subset of histograms that will not be too difficult to be understood.
A.M. Could you tell us more about the history of AMORE. How was it developed and how many people were actually involved?
B.v.H. When I arrived in ALICE there was a prototype. I took it over and put it into production and since then I developed it. At that time I was alone on this project but later on I have had the chance to have Adriana Telesca working with me on this project. The code is mostly written in C++ in conjunction with the framework ROOT and a certain number of scripts and side tools have been developed in other languages. The main idea is that I develop a general framework and then physicists from the different detectors work with me to write the algorithms that will do the actual analysis. As I am not a physicist I have to collaborate with them to produce the final product.
A.M. Can you give us more details on how AMORE works?
B.v.H. AMORE picks events from the data acquisition chain and feed them to the numerous detectors modules. The criteria and the algorithms used for their analysis are decided by the Data Quality Monitoring responsible for each detector’s team. They come with their own ideas and then I give them instructions on how they could proceed and implement them. After that the detector team actually writes the code and the checks that ensure that the quality of data fits to their criteria. Finally, I validate their code and possibly help them improve or fix some problem before it goes into production at P2.
As soon as a run begins we start collecting statistics and very quickly we have the first results that allow the DQM shifter to decide whether the data is correct. It can vary a lot because in some cases you have noise or something wrong with the beams. However, it is important to be able to detect these faults as fast as you can.
AMORE gets data from different sources, one being the Detector Algoritms (DA). Here the SPD DA allows the shifter to check the Vertex.
A.M. And when you say “very fast” what is the scale to define it?
B.v.H. What is important is to collect and analyze as many events as possible. This is what really matters, to have a big sample. Afterwards it doesn’t take very long to analyze these events. The longest would be 1 sec or at least in this order of magnitude. It’s fast and the reason is that we do not do excessively complex things but rather focus on collecting a lot of statistics and we run efficient software on powerful machines. It plays role to be able to analyze 1000 or even more events for the results to be meaningful. Usually within ten minutes you understand whether something is going wrong or not. However, there are times when after six hours of run things might go ugly, for example you can have one of the detectors decalibrated and so on.
A.M. Where does the data analysis take place?
Q.M. At Point 2 there are several computer rooms and the DQM shares CR1 with the DAQ. It is a good thing as we need to pick data samples from the Data Acquisition chain. There we have many powerful computers and this is where the actual analysis is taking place.
A.M. Do you collect a different set of data from those taken by the experiment?
B.v.H. Actually it’s the same data but we take only samples. So you have the data going through data acquisition and we pick only some events (ideal case would be to pick all of them). The monitoring must not interfere with the data acquisition nor slow it down. This is why we only take a few samples the size of which depends on the running conditions at LHC.
A.M. Do you still modify and change things in AMORE?
B.v.H. Yes. We need to change things at the level of the framework from time to time and down in the detector modules quite often, including the part of the analysis that might reside in AliRoot. Of course for some detectors there are only minor changes such as changing some thresholds or tune certain algorithms. On the other hand for certain detectors that are more complex, e.g. ITS or TPC, the code and the thresholds regularly evolve.
Moreover, we use a lot of software that has been written for AliRoot and every time AliRoot is updated we need to make sure that it is compatible and works well with AMORE. This is something that we do regularly and we really need to be careful. AMORE interacts with many different systems not only from AliRoot but also receiving data from what we call detector’s algorithms – pieces of software that haunt in the DAQ - that do some data analysis in view of producing calibration data. We also take data from DCS, the Data Control System. So AMORE lies in the middle of these systems, collecting results and producing new data that are used not only by the DQM Shifter but also from other software.
Plot generated by one of the DAQ AMORE modules to monitor the data size of the detectors over time.
A.M. When did you decide to move into computer science and did you imagine that you would move into the world of particle physics?
B.v.H. At the end of high school I had to decide about my next step in the University. I hesitated between doing medicine or something more technical. In the end , I decided to go to the EPFL in Lausanne to study computer science. I was already developing software for myself and I was enjoying it a lot, thus why not turn my hobby into a career? When I finished my studies it was clear that I would continue doing this.
Now regarding your second question I think that mixing with physicists and other scientists is one of the nicest aspects here. As I am from Geneva – I must be the only Swiss in ALICE! - I thought that it would be wonderful to work in a place like CERN. I had this idea from a young age and I must admit that being here just met all my expectations. Especially in ALICE, I enjoy the collaboration with people from different backgrounds and cultures. In my group, although we are all mostly doing programming, we are half physicists, half computer scientists or engineers and hence we bring something different depending on our qualifications and experience. That’s something that I really enjoy at CERN.
A.M. And your future plans?
B.v.H. I have started working with my colleague, Adriana Telesca, on the documentation of the DAQ projects. At first sight it seems trivial. However the variety of information and audiences make things a lot more complex than one would think. For example, we have the shifters working at P2 seeking basic information on how to operate, the collaborators who need to install DAQ software in their laboratories, and finally the members of our own group as we need to share knowledge and information. Our aim is therefore to sort, reorganize and filter all this knowledge and information.
We are also working on a future paper that will describe the data acquisition as it is now before applying major changes during the long shutdown. So we want to publish this paper in a peer-reviewed journal and I, as “editor”, am currently coordinating the writing of this paper.
I have also started the training to become a guide at CERN because I think it is important and exciting to share with people what we do here at CERN and in ALICE.