My department specializes in converting customer data into our database schema so that they can use our software.

Right now, we have C# applications that take an IDataReader (99% of the time it is a SqlDataReader), perform some cleaning and mapping, insert it into a DataRow object, and then use a SqlBulkCopy to insert it in to our database.

Sometimes (especially when the source database contains images as varbinary objects), this process can really bog down with a SQL transfer from the server to the app to just then turn right around and go back to the server.

I feel that if we re-wrote some of the conversions as SSIS packages it could speed things up a lot. However the biggest stonewall I keep running into is when my boss, in Not Invented Here fashion, pushes back and says "What if Microsoft drops support for SSIS? We will have all this obsolete code and we will be screwed."

This is not the first time that I have hit a "What if they remove that feature...?" answer from my boss. I don't have the time to write the conversion the old way, self-teach myself SSIS, and also write it the new way to demonstrate/test the benefits (None of us have used SSIS so there would be a period where we would have to learn how to use it).

What should I do in this situation? Stop pushing the new technology? Wait till he leaves the department (I am the 2nd most Sr. person in the department after him, but it could be years before he quits/retires)? Find a new way of getting him to stop being afraid of 3rd party tools?

I'll take a crack at this from a managerial standpoint, but keep in mind that I'm aware I don't have all of the details. I'll summarize what I see:


Mid-level developer, we'll call him "Scott", recommends a rewrite of legacy code into SSIS to improve the performance of important process ProcessA.
ProcessA is currently behaving in a functioning state with no known major issues.
ProcessA is written with proprietary tools using common or potentially tribal knowledge to maintain.
Recommendation to rewrite will require new tools to support.
Current staff has no documented experience/knowledge with the new tools.
New tools are relatively recent replacements to older tools, and support for these tools may change reasonably within 4 business quarters.


From this perspective, all I see is a large outlay of money on the company's part to improve a process that isn't broken. From a technical standpoint I can see the appeal, but when you get right down to it, it just doesn't make business sense to make this change. If you had staff on hand with documented experience with SSIS and benchmarks to show massive improvement to this process (keep in mind, massive improvement MUST equate to $$$), the outcome might be a little different. As it stands now, though, I'd have to agree with management (somewhere a tree just died).

If you want to foster the adoption of SSIS and potentially lead into this refactor, you need to gain this experience and training with smaller, less important projects. Provide benchmarks and support for SSIS, and make sure that all of the infrastructure and support is in place before management will even consider making the change. Once you have the tool in use elsewhere, people on the team experienced with its usage, and a business "comfort" factor that support won't change and uproot everything, you will be more likely to sway someone to your viewpoint. Without those, you're barking up the wrong tree with that argument.

As stupid as it sounds, sometimes the "best" way isn't the best way.

Edit: In response to some updates to the question, I'll post a slight modification to my answer.

If the process is experiencing distress of some kind, rewriting it will still be a costly venture. You may want to consider what the cost of fine-tuning the existing code would be against rewriting the package. Consider the impacts not just to software but to any human interfacing processes. When attempting to get management buy-in to a rewrite, it still will always come down to the money. Unless you can show that the current distress is costing money now or will become large in the aggregate, management will still not see the benefit. This cost may not necessarily be financial in nature. If the slow down compromises a system causing downtime, introduces intrusion vectors, or some other "hard to quantify" symptom, you may need to find a way to translate that problem into a monetary risk equivalent. An intrusion vector, for instance, may lead to an intrusion which could result in lost, stolen, or corrupt data. The company could lose reputation or may fail a necessary security audit. The key is to get the manager in question to recognize the quantifiable benefits of the change.
