<h1>Software craftsmanship</h1>

<p>As professionals, we need a set of ethics (ethics is not the same as morality). Medics have ethics, even lawyers have ethics. They can be struck off from their profession if they do not stick to them. If we do not create our own, then they will be imposed on us (see talk by <a href="https://www.youtube.com/watch?v=ecIWPzGEbFc" rel="nofollow noreferrer">Uncle Bob Martin</a>).</p>

<p>In some ways it does not matter exactly what they are. You could have a class discussion. You could include those proposed by others. There are many points of view. A few contradict, but many do not.</p>

<p>I have included those proposed by Uncle Bob et al.</p>

<h2>Agile manifesto</h2>

<p>We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:</p>

<ul>
<li>Individuals and interactions over processes and tools</li>
<li>Working software over comprehensive documentation</li>
<li>Customer collaboration over contract negotiation</li>
<li>Responding to change over following a plan</li>
</ul>

<p>That is, while there is value in the items on
the right, we value the items on the left more.</p>

<h2>Software craftsmanship manifesto</h2>

<p>As aspiring Software Craftsmen we are raising the bar of professional software development by practicing it and helping others learn the craft. Through this work we have come to value:</p>

<ul>
<li>Not only working software, 
but also well-crafted software</li>
<li>Not only responding to change,
but also steadily adding value</li>
<li>Not only individuals and interactions,
but also a community of professionals</li>
<li>Not only customer collaboration,
but also productive partnerships</li>
</ul>

<p>That is, in pursuit of the items on the left we have found the items on the right to be indispensable. </p>

CC BY-SA 3.0
2017-08-27T12:03:41.137Z
2017-08-27T12:03:41.137Z
ctrl-alt-delor
<p>I was recently watching videos of <a href="https://libreplanet.org/2017/" rel="nofollow noreferrer">Libra Planet 2017</a>. A lot of their talks covered ethics. </p>

<p>There seems to be a <strong>lot</strong> of ethical considerations in software. They did not cover everything, but here is a summary of some of the topics:</p>

<ul>
<li>Jobs.</li>
<li>Security.</li>
<li>Privacy.</li>
<li>Equal rights / user Disability.</li>
<li>User rights: Right to use, right to study, right to distribute, right to change …</li>
<li>Algorithmic bias: e.g. search, selection (for jobs), crime detection.</li>
<li>Law: The good, the bad, and the ugly.</li>
<li>Politics.</li>
<li>Free speech: Software that you can go to jail for talking about (in the USA).</li>
<li>Workers(programmers) rights and responsibilities (guilds and trade bodies).</li>
</ul>

<p>Some things not covered.</p>

<ul>
<li>Safety.</li>
<li>Artificial Intelligence.</li>
</ul>

CC BY-SA 3.0
2017-10-27T08:59:03.200Z
2017-10-31T08:40:12.697Z
2017-10-31T08:40:12.697Z
ctrl-alt-delor
<h1>The additional dangers of software over other engineering fields.</h1>

<h2>Who is in charge</h2>

<p>All software is a form of artificial intelligence, by this I do not mean artificial consciousness (e.g. skynet, I Robot, ExMachina, robby, and every other robot from the movies). </p>

<p>I think that the probability of artificial consciousness arising soon is very low. However some form of artificial intelligence is in every bit of software. The danger is not the artificial intelligence, but who controls it. A lot of software to day is controlled by big corporations. Users of the software are not free to: choose how they use it; study how it works; make changes to it; share it with their friends. </p>

<p>Therefore all software should be Free Software. </p>

<p>I am not referring to Open Source. Although (Software that is Free Software) ≈ (Software that is Open Source). I am referring to the concept of Free Software. The freedoms, for it is the freedoms that we need. These freedoms are also in the Open Source definition, but are usually over looked.</p>

<p>I am also not saying that all Free Software is ethical. There are many sides to being ethical. There may also be proprietary software that is more ethical than a particular bit of Free Software. <strong>Free Software is necessary but insufficient, to being ethical.</strong> Just as obeying one of the 10 commandments is necessary but insufficient, to being good (Even obeying all of the law is insufficient).</p>

<blockquote>
  <p>“All these [laws] I have kept,” the young man said. “What do I still lack?”.
  He answered, “If you want to be perfect, go, …” </p>
</blockquote>

<p>What I am saying is about the ethicalness of the software, not of the user.</p>

<p>Free Software has nothing to say about money (except to say that you must be free to charge money for it). </p>

<h2>Artificial consciousness</h2>

<p>Just a small paragraph to counter what I said above.</p>

<p>If some one interfaces a biological brain to a computer, and gives it power. That is the ability to do harm, then it probably will. Any disembodied brain will have no empathy. </p>

CC BY-SA 3.0
2017-08-02T19:27:10.570Z
So the only ethical software is "free" software, where "free" is defined by Stallman's turgid GPL. I disagree.
President James K. Polk
CC BY-SA 3.0
2017-08-02T19:49:03.080Z
@JamesKPolk but not all Free Software is ethical. I have amended answer to address this. You are saying that because software A has some aspect of ethical then it **is** ethical. And You are thinking that I am saying that all Free software is ethical. It is not that simple.
ctrl-alt-delor
CC BY-SA 3.0
2017-08-02T19:51:47.973Z
@JamesKPolk The Free Software Definition defines what Free Software is. The GPL is but one implementation. You should give them a read, you will find that the language is very simple and short.
ctrl-alt-delor
CC BY-SA 3.0
2017-08-03T08:15:48.373Z
-1 if I could downvote on this site. This doesn't seem to have anything to do with ethical software.
David Mulder
CC BY-SA 3.0
2017-08-03T08:38:01.047Z
Why would a disembodied brain have no empathy? Are you suggesting empathy comes from the liver or what? It's not a separate system that restricts the brain, it's part of the brain. Not that I'm saying it's a good idea trying to model a true artificial intelligence after the ridiculously flawed machine built by natural selection :)
Luaan
CC BY-SA 3.0
2017-08-03T09:37:27.053Z
@Luaan No I was thinking that empathy comes from your interaction with others. But I would leave it as a discussion for the students, to decide if they think I am correct.
ctrl-alt-delor
CC BY-SA 3.0
2017-08-03T11:40:06.073Z
@nocomprende https://www.youtube.com/watch?v=IHyH3MPgZDo
ctrl-alt-delor
CC BY-SA 3.0
2017-08-02T19:19:41.203Z
2017-08-03T09:56:04.473Z
2017-08-03T09:56:04.473Z
ctrl-alt-delor
<p>There are certain unsolvable problems (I am not referring to NP), that seem solvable, and an apparent solution can be made and deployed. These systems can be used, but do not fulfil there requirements (security, privacy, …).</p>

<p>This differs from the case where the software lacks these features, because of poor workmanship (also an ethical issue), in this case it can not be done. I turn down the work, because it is impossible. The client thinks they need to find someone that is more clever, that can do it. Eventually they find someone that has no idea of the problems, that agrees to do the project. If the new supplier realises that it can not be done, and tells the client, then the process repeats, else a faulty product is delivered.</p>

<p>An example is electronic voting.</p>

<p>It is impossible to create an electronic voting system that meets all of the requirements. If we just have 2 of the requirements for a voting system.</p>

<ul>
<li>Anonymity</li>
<li>Trustable</li>
</ul>

<p>Then we can not produce an electronic system, that can meet just these requirements. There are also other requirements, thus it is even harder (in the higher than infinity sort of way (∞+1 = ∞), therefore we don't need to look at the other requirements, to see how hard it is).</p>

<p>How would you audit this system? You could check its source code, to see that it does the right thing (if you can read it, so every one involved would have to have a good enough level of CS, and know the language). Oh but how do we know that the system is running this code.</p>

<p>Every single solution to this problem, has a counter measure ( a dis-proof ).</p>

<p>This has not stopped these things from being deployed many times, and in countries that are considered to have a well run voting system.</p>

<hr>

<h1>What is needed</h1>

<p>A box, that you put all of your votes into, and out comes a count for each candidate. </p>

<h1>Possible solutions</h1>

<p>How do you know that the output is correct. Ah that is easy, you produce an audit trail. But now we have no anonymity. So we add encryption. But for it to be audit-able, anonymity can still be broken. (If you can check your own vote, then I can check your vote (if I torture you long enough, then you will give up your key).</p>

<h1>A solution that works, but does not use a computer.</h1>

<p>The many boxes
  - you walk into a room (box1), and show your ID, your name is removed from the list, and you are given a voting sheet.
  - you walk into a subroom (box2), and make your mark on the sheet.
  - you then fold(box3) the sheet.
  - you leave box2 and walk to a locked box (box4), and place your sheet into it.
  - at all times observers (from all interested parties) are walking around box1, to check that the rules are being followed.
  - at the end box 4 is sealed.
  - latter box 4 is opened and the sheets counted. With lots of people watching. A set of totals is produced and announced. The sheets are put back in box 4, sealed and locked.</p>

<p>I have probably missed some detail, but this solution cannot be implemented electrically, because there is no way of putting people into the electronic boxes to observe. </p>

<h1>Some videos on this</h1>

<p><a href="https://www.youtube.com/watch?v=w3_0x6oaDmI" rel="nofollow noreferrer">The problem</a></p>

<p><a href="https://www.youtube.com/watch?v=BYRTvoZ3Rho" rel="nofollow noreferrer">Some solutions</a> I found a problem with all of these solution, as I was watching it (Did not have to sit and think for days).</p>

<p><a href="https://www.youtube.com/watch?v=ZM-i8t4pMK0" rel="nofollow noreferrer">Other stuff</a></p>

CC BY-SA 3.0
2017-08-03T08:12:25.810Z
You make it sound like paper voting somehow has less issues with trustworthiness. Technical solutions could theoretically improve trustworthiness (as you increase the number of people who would all need to be 'in on it'), but it's a very complex issue. Anyway, -1 if I could vote down on this site.
David Mulder
CC BY-SA 3.0
2017-08-03T09:42:21.533Z
@DavidMulder So if I provide a system, for an election. It checks each voters ID. We could ensure anonymity by never ever ever, connecting it to a network after it has been used. However there is no way for the voters to know. Then it takes your vote, and reduces it to a count for each candidate. How will you as a candidate ensure that it does this correctly ( It could just make up the numbers, in favour of the candidate that it wants to win).
ctrl-alt-delor
CC BY-SA 3.0
2017-08-03T12:37:51.007Z
@nocomprende it has already been solved for paper based system, but you may have improved it. However I see no need to use dollar bills: once country elects an untrustworthy precedent, the dollar bill would be as trust worthy as raffle tickets. But still an improvement, as it allows each voter to verify there part. The existing system focuses on the candidates and parties verifying the system. Can you see why it is impossible to implement using a computer system?
ctrl-alt-delor
CC BY-SA 3.0
2017-08-03T13:09:20.217Z
While the answer has some merit in the context of the question (for example as a teaching aid to show layers of complexity for the use case of electronic voting systems), I would be *really* reluctant to make the  absolute claim that it is prinicipally impossible. Cryptography gives us tools similar to the pen&paper, anonymous envelops, tokens, booths...; without having really analyzed it, I could see possible solutions. Obviously, part of it would be that local machines (+ their software) would be assumed to be compromised, and such compromising must be detected by the "crypto" part.
AnoE
CC BY-SA 3.0
2017-08-03T13:23:54.213Z
@AnoE because you can not trust or audit the computer, you need end to end checking (crypto). Now get this to work with anonymity and inability to show how you voted. The some solutions video talks about these, but I see problems with all of the solutions (and I am to that clever). This is without having to resort to “How does my mum verify a cryptographic hash by hand (because she dose not trust the computer).”
ctrl-alt-delor
CC BY-SA 3.0
2017-08-03T13:30:29.500Z
@richard: my point is not that I have outlined a working electronic voting system in a SE comment. ;-) My point is that is incredible hard to argue that "solving XYZ is not possible and will never be possible". It is absolutely fine to argue that todays solution to XYZ suffer problems A, B, C and that nobody has yet found a working solution, but your answer seems to say that it is theoretically and in all principles impossible to solve the problem.
AnoE
CC BY-SA 3.0
2017-08-03T17:27:35.357Z
@AnoE My point is that no implementation can ever be found, because the way it is implemented is impossible. You can be a black box analysis of the requirements. The requirements contradict: Actually the real requirements are very hard, and only just possible, but if you say that you must have a computer, you add an implied requirement that no one can observe. With this no requirement there becomes a contradiction.
ctrl-alt-delor
CC BY-SA 3.0
2017-08-03T17:32:34.360Z
@AnoE  I just looked up homomorphic encryption, just to see what it was, because there is a clam that this solves the E-voting problem. I predicted that it must have the property of “whoever can read the result, can find out who you voted for”, for it to do what it was claimed to do. I don't understand the math of the fully homomorphic systems, but their usage manuals confirmed my fear. It does not matter what happens in the box, it must have these properties, therefore in is of no use in implementing an E-Voting system. It looks like it may have other uses though.
ctrl-alt-delor
CC BY-SA 3.0
2017-08-27T12:13:45.850Z
Since my lost comment on this, I heard a talk by a women that has her Phd + in electronic voting systems, she has several **mathematical proofs** that they can not do what they need to do. One of her colleges invited a group of “young people” to hack some real voting machines (manufacturers claimed 100% secure, and trustworthy) , in a 1 or 2 week session. She was sceptical that there would be time, to find any thing. First exploit found in two minutes. All compromised by end of exercise. Could be that all current systems, are made by cowboys. But see proofs, or linked video for general case.
ctrl-alt-delor
CC BY-SA 3.0
2017-10-27T19:31:04.463Z
I'm not seeing why this fits under "*ethical practices*".
Peter Taylor
CC BY-SA 3.0
2017-10-27T19:55:30.823Z
@Peter Taylor Is it ethical to have the people use voting systems, that do not do what they say they do. That do not do what the voting laws say they must do.
ctrl-alt-delor
CC BY-SA 3.0
2017-08-02T19:34:02.510Z
2017-10-29T14:20:15.703Z
2017-10-29T14:20:15.703Z
ctrl-alt-delor
<p>I'm going to begin by quoting Ken Thompson's Turing Award Lecture "Reflections on Trusting Trust" (<a href="https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf" rel="noreferrer">link</a>).</p>

<blockquote>
  <p>To what extent should one trust a statement that a program is free of
  Trojan horses? Perhaps it is more important to trust the people who
  wrote the software.</p>
</blockquote>

<p>In my mind the first and most foundational lesson to impart is this: <strong>software, computers, technology -- these things are more about people than about code.</strong> There is a trust from the end-user, whoever that user may be, that must not be violated. Users are not means to an end; they are the end. The code is the means.</p>

<p>I am reminded of the <a href="https://en.wikipedia.org/wiki/Categorical_imperative#The_Second_Formulation:_The_Formula_of_Humanity" rel="noreferrer">Second Formulation of Kant's Categorical Imperative</a>:</p>

<blockquote>
  <p>Act in such a way that you treat humanity, whether in your own person
  or in the person of any other, never merely as a means to an end, but
  always at the same time as an end.</p>
</blockquote>

<p>In terms of integrating ethics, the way to start in my mind is discussion. Early and often. Share news stories. Discuss implications of new apps students download. The first 12 minutes of <a href="https://www.youtube.com/watch?v=8Ba7SgCN2XY" rel="noreferrer">this CS50 lecture</a> cover the the Volkswagen/EPA scandal from 2015 and mention Ken Thompson's lecture. It is essential that awareness is built right from the beginning that software is everywhere and there needs to be an ethical code to guide that development. In my mind, while I wouldn't consider myself Kantian, I do think his point about means and ends is essential.</p>

<p>The AP Computer Science A curriculum outlines the following requirements under the major topics of the course:</p>

<blockquote>
  <p><strong>VI. Computing in Context</strong></p>
  
  <p>An awareness of the ethical and social implications of computing
  systems is necessary for the study of computer science. These topics
  need not be covered in detail, but should be considered throughout the
  course.</p>
  
  <p>A. System reliability</p>
  
  <p>B. Privacy</p>
  
  <p>C. Legal issues and intellectual property</p>
  
  <p>D. Social and ethical ramifications of computer use</p>
</blockquote>

<p>Similarly, in AP CS Principles, students, as a part of the Explore Task, have this on their rubric for the assessment:</p>

<blockquote>
  <p><strong>Analyzing Data and Information</strong>: Identifies one storage, privacy,
  OR security concern. Explains how the concern is related to the
  computing innovation.</p>
</blockquote>

<p>The moral of the story here is that ethics must be discussed with students as soon as possible whether it is a high school AP course or an introductory computer science course at a university.</p>

<p>This idea of <strong>trust</strong> must be imparted to students, and it must be emphasized that the world of software engineering and computer science in general is not in a moral vacuum. It does no good to address a particular issue, say data privacy, without first contextualizing it within a larger ethical system, one that sees each user as an end, not a means, and as someone who is placing trust in you the developer.</p>

CC BY-SA 3.0
2017-08-02T16:42:52.413Z
@nocomprende I don't disagree. I am reminded of the great wisdom of Dr. Ian Malcolm from _Jurassic Park_: "Your scientists were so preoccupied with whether or not they could that they didn't stop to think if they should." People need to "stop to think." Instilling that "need": "Ay, there's the rub."
Peter
CC BY-SA 3.0
2017-08-03T14:19:10.687Z
The only person I would trust is a madman who would rather voluntarily live in poverty than allow himself to be pressured. Well, since everyone else by definition can be pressured and bought to sneak in bad stuff in the code which therefore eradicates any trust.
mathreadler
CC BY-SA 3.0
2017-08-03T15:58:10.710Z
@nocomprende no idea what this "Tao" says or who he is. But there do definitely exist enough ways to pressure most people to stop trusting the concept "trust" altogether.
mathreadler
CC BY-SA 3.0
2017-08-03T20:04:17.303Z
@mathreadler. Even the *Road Warrior* world had *some* trust. You have to get to *Lord of the Flies* to go without it. But perhaps you only mean "trust but verify".  Also, conversations to the classroom, please: https://chat.stackexchange.com/rooms/59174/the-classroom
Buffy
CC BY-SA 3.0
2017-08-02T16:36:51.190Z
2017-08-02T16:36:51.190Z
Peter
<h1>None.</h1>

<p>Yes, a bold statement, literally. So let me elaborate further.</p>

<p>You cannot cover all potentially relvant aspects of "ethical practices" in a computer science course. Assuming that this is a general, basic course, you simply cannot foresee the potential ethical questions that will arise for these students. One of them may work in the IT department of a hospital. He should not sell patients data to a pharma company. One of them may work in a bank. He should not disclose or manipulate the financial status of customers. Another one will design brain-computer interfaces. He should not use his abilities to alter the behavior of the subject in a non-ethical way (<em>whatever this means!</em>). Another one may be responsible for implementing the emergency handling of a self-driving car, and the ethical implications of that are currently the subject of deeply philosophical (and still open) discussions</p>

<p>Related to that is another problem: You can hardly ever foresee the ethical implications that your work may have. Although this example has been used ad nauseam (and may be considered as some sort of a fallacy), I like the comparison of computer science and a sharp knife: You may use it to stab someone, or to perform a life-saving surgery: The data analysis software for weather forecasts may be used to compute questionable credit scorings. The image analysis software that you're implementing to sort holiday photos may be used to identify people in surveillance camera images. </p>

<p><sup>(This has a huge potential for abuse. The fact that people could now argue that this "helps to catch terrorists" shows how difficult this topic is. The ethical discussion <em>itself</em> is beyond the scope of this answer and this site. In fact, the examples here should only show <em>exactly</em> this: You cannot summarize "ethics" on a few PowerPoint slides...)</sup></p>

<p>And finally, assuming that the students are in their 20's: It's too late. You will not convert an (ethically) "bad" person into an (ethically) "good" person by listing things that they could do but should not do. Acting responsibly (in an ethical sense) is nothing that can (or should) be tought alongside the basics of abstract data types, computer architecture or object oriented design...</p>

<hr>

<p>To summarize, although most of the downvoters will likely not read up to this point: A computer science course is not the place to <strong>train</strong> people in <strong>ethical practices</strong>, as stated in the question. </p>

<p>What you may (or even <strong>may have</strong> to) teach is <strong>awareness</strong>. People should not act unethically (or just aid others in acting unethically) due to a lack of knowledge. Computer scientists, particularly in their later (practical) roles, for example, as IT administrators, have a lot of power, and thus, a lot of <em>responsibility</em>. They should be aware of the fact that their knowledge or work (or a moment of carelessness) may literally destroy lifes. </p>

<p>This awareness about the implications of their work is unrelated to ethics. Whether or not they use their powers in an ethical or unethical way is nothing that you can "teach" them.</p>

CC BY-SA 3.0
2017-08-03T17:31:56.737Z
You seem to be contradicting your bold statement (I noticed the pun). If I understood correctly, you are saying that *because of the serious dangers that unethical IT workers might pose*, they **shouldn't** be taught not to do those things? Even if one cannot "foresee" the ethical implications work might have, I doubt it will ever be considered ethical to disclose secret information which leads to **direct** harm to others. That can be taught, and [You don't need the gift of foresight](http://www.quotes.net/mquote/120399) to know what sort of ethics would be useful for your students.
ItamarG3
CC BY-SA 3.0
2017-08-03T18:52:55.373Z
@ItamarG3 The point is that *teaching* (and thereby to some extent *"defining"*) ethics is not sensibly possible in a CS course. Regarding your example: Disclosing information that may harm others could be "OK" if the "other" person is a terrorist who plans an attack. Whether this is unethical can neither sensibly be discussed here nor in a CS course. On the other hand, what you **can** teach is, for example: **Which** measures may prevent unauthorized access to personal data, or which pieces of information can be combined in order to identify someone, from a purely technical point (...cont'd)
Marco13
CC BY-SA 3.0
2017-08-03T18:57:51.750Z
(also @nocomprende ) : To **judge** whether something is ethical/unethical, right/wrong, good/evil depends on the context. Although you can [define a motto, like Google did](https://en.wikipedia.org/wiki/Don%27t_be_evil), this won't mean anything. Although there certainly is some consensus about what is "right" or "wrong" (regarding WW2, most people will call it "wrong"), **teaching** this is totally unrelated to the contents of basic CS courses. Or another summary: You should not teach *ethics*, but the (technical) points of the work that may have *implications* for ethics.
Marco13
CC BY-SA 3.0
2017-08-03T20:10:25.723Z
Let me suggest that ethics isn't an infinite set of "rules": if A then B, but a set of principles and dispositions to act in a certain way. Physicians have been trained "Do no Harm" for quite a long time, though the quote is attributed to Florence Nightingale. Too many rules: yes - don't even try. Infinitely applicable principles: yes
Buffy
CC BY-SA 3.0
2017-08-04T04:43:30.960Z
"... assuming that the students are in their 20's: It's too late. You will not convert an (ethically) "bad" person into an (ethically) "good" person..." This assertion is [not supported by the research](https://www.scu.edu/ethics/ethics-resources/ethical-decision-making/can-ethics-be-taught/) about ethics and morality. We don't even seriously *start* developing a cohesive moral perspective until our 20s, and this process continues well into our 30s.  The very late teens and the early 20s are prime time in the brain for discussing moral and ethical action.
Ben I.
CC BY-SA 3.0
2017-08-04T12:51:49.483Z
@BenI. It's not tooo surprising that the ethics department of a university claims that ethics can be tought ;-) But that's not the point. The point is rather that this ethics department will not teach the students that ethical implications can be modeled with binary logic on NAND gates. One could blatantly say: "That's off-topic". Again: CS students should be aware of their responsibility and how their work may affect the lifes of others. But trying to *teach* them what is "right" or "wrong" (and for which reason) seems pointless for me, as these are subjective, open-ended discussions.
Marco13
CC BY-SA 3.0
2017-08-04T21:10:39.100Z
@nocomprende The trolley example is *very* obvious, and I referred to it in the answer. There are dozens of examples that are similarly (although not so obviously) subtle. To put it that way: The CS/engineer will not decide about the track that the trolley will take, but he will have to make sure that there is a lever to switch the track in the first place, and be aware that, if this lever does not work, something will happen what *others* defined to be ethically wrong (regardless of what this is).
Marco13
CC BY-SA 3.0
2017-08-03T16:59:00.520Z
2017-08-03T16:59:00.520Z
Marco13
<p>Raise awareness of the issues, discuss historical (Therac 25 is an old but highly interesting case of software bugs directly causing people to get hurt) and current cases (EU/US safe harbor and followups - there is so much potential to go in-depth about what data protection means and what the risks are!), and maybe make the legal situation (especially international law!) part of the curriculum ... but don't try to "train" people towards any ethical viewpoint. The latter is in the end straight manipulation no matter how well intended, and something people should if at all be trained to guard against.</p>

CC BY-SA 3.0
2017-08-03T01:55:57.807Z
2017-08-03T01:55:57.807Z
rackandboneman
<p>Teach them to be cautious. This may not sound like ethics, but actually it has a significant ethical effect. </p>

<p>People - especially young men - who are very confident try all kinds of things, including unethical ones. Their confidence is often misplaced. Computers will not forgive you because you are young and charming. They'll happily retain the evidence of wrongdoing, and it's very hard to clean it off them completely. </p>

<p>Misuse of computers tends to be discovered. Unlike people, computers do not cover up evidence of deception, because they're incapable of being embarrassed. </p>

<p>Besides, caution is sound engineering practice. You get two useful results from one set of teaching. </p>

CC BY-SA 3.0
2017-08-05T17:53:18.737Z
2017-08-05T17:53:18.737Z
John Dallman
<blockquote>
  <p>What are the first steps a teacher should take in integrating ethics into the curriculum?</p>
</blockquote>

<p>I would begin with practical, everyday topics which every student can relate to, and which are (should be) absolutely practically relevant in todays world from day #1 for every developer. Maybe based on current newspaper-level discussions.</p>

<ul>
<li>Privacy (being able to stay private when using an application, controlling who can see what you do)</li>
<li>Data security (being able to protect your own data, e.g. in the context of a business)</li>
<li>Personal data (being in control of data related to yourself as a human being; related but not equal to Privacy)</li>
</ul>

<p>You should easily be able to find an abundance of concrete examples, let me give you some that come to mind:</p>

<ul>
<li>Privacy: communicating with someone while being sure that nobody can read your chat. May it be your spouse (say when organising an expensive birthday present for them), your system administrator (when executing your function as a manager to discuss your subordinates with your boss, etc.), or the general public (obvious reasons...).</li>
<li>Data security: Keeping contract offers away from competitors.</li>
<li>Personal data: Keeping preferences, habits, interests etc. from prospective employers or insurance companies.</li>
</ul>

<p>I would ask the pupils to come up with their own, they should be able to find a lot of different things.</p>

<p>This, then leads to your question: it is ethically important for everybody involved in the creation of software systems to further these things. I.e., not to be laizzes-faire about them, not to prioritize them lowly when budget or time pressure arises, not to actively violate them; on the other hand, to constantly remind/teach the stakeholders, who may not have an understanding of these matters, and so on.</p>

CC BY-SA 3.0
2017-08-03T14:52:47.190Z
@nocomprende: absolutely. Not only the Web, but the Internet as a whole. I was active in Fidonet, Usenet etc.; one day DejaNews, then Google, just slurped in the whole Usenet history, for eternity. Before that, Usenet was kind of temporary. That was kind of *the* eye opener for me; before that nobody gave a d*mn about what they posted (private information, etc.); afterwards not so much. If I check Google Groups now, I easily find all my post history from the early 90's, including real names, mail addresses, phone numbers in the signature and such things...).
AnoE
CC BY-SA 3.0
2017-08-03T13:28:28.437Z
2017-08-03T13:28:28.437Z
AnoE
<p>The kinds of things that people with technical training can do have societal impacts, both for good and evil. Programmers are building the future. One future nightmare is <a href="https://en.wikipedia.org/wiki/Skynet_(Terminator)" rel="noreferrer">Skynet</a>. </p>

<p>Many problems in the world have been caused by scientific hubris as well as the <em>Fallacy of the Last Move</em>, a concept from Game Theory as it applies to Arms Control and other strategic thinking. </p>

<p>Unbound technologists, especially if amoral, can be very dangerous to the earth and its inhabitants. </p>

<p>So, it seems that future Software Developers need some training in ethical concerns. What, to you are the most important issues and where in the curriculum should they be taught? </p>

<p>Note that ethical issues can be simple or complex. Even something as simple a accepting buggy software can have ethical implications and such lessons can be taught in early courses. </p>

<p>At the other end of the scale is the question of how much a person owes to their employer vs to humanity as a whole. </p>

<ul>
<li><p>What essential ethical practices do students need to be trained in? </p></li>
<li><p>What are effective ways of integrating ethics into the curriculum? </p></li>
<li><p>What are the first steps a teacher should take in integrating ethics into the curriculum?</p></li>
</ul>

<p>A general discussion of Computer Ethics is: <a href="https://plato.stanford.edu/entries/ethics-computer/" rel="noreferrer">https://plato.stanford.edu/entries/ethics-computer/</a></p>

CC BY-SA 3.0
2017-08-02T15:47:32.890Z
I think faulty engineering is a likelier ethical concern than Skynet.
Ellen Spertus
CC BY-SA 3.0
2017-08-02T19:52:22.633Z
The BCS Code of Conduct might give some insight into this issue; http://www.bcs.org/upload/pdf/conduct.pdf
Mikael Patel
CC BY-SA 3.0
2017-08-02T21:03:33.773Z
Speaking as a machine learning guy, I *wish* creating AIs was actually easy enough that a rogue programmer creating Skynet was a realistic concern.
Ray
CC BY-SA 3.0
2017-08-02T22:00:13.240Z
I highly recommend "Ethics and Technology: Controversies, Questions, and Strategies for Ethical Computing" by Herman T. Tavani.
Alex
CC BY-SA 3.0
2017-08-03T08:04:26.700Z
Some good case studies, would be Uber (http://www.bbc.co.uk/news/business-36843386), Ashley Madison (The site itself, and the subsequent information disclosure), Encrypted Messaging vs National Security (http://www.express.co.uk/news/world/834597/WhatsApp-Facebook-removing-encryption-would-make-extremists-harder-track)
JeffUK
CC BY-SA 3.0
2017-08-03T16:47:59.290Z
@nocomprende: as someone that survived lymphoma I really, really disagree with you.
Rui Barradas
CC BY-SA 3.0
2017-08-03T18:35:07.357Z
@nocomprende: I still believe that the question cannot be stated like that. Lack of food and water are about lack of social structures, such as Democracy, that establish societies where people's wellfare is guaranteed. The cure for diseases is a completely different thing. Of course you could argue that sciencific and technological development are only possible where those basic problems are already solved, but if they are, then we will not have to choose. I think that your dichotomy is not well put.
Rui Barradas
CC BY-SA 3.0
2017-08-03T20:00:39.123Z
Conversations like the above are appropriate in the classroom. https://chat.stackexchange.com/rooms/59174/the-classroom. Less so in comment streams.
Buffy
<curriculum-design><best-practice><ethics>
What ethical practices do students (as potential software engineers) need to be trained in?
# What ethical practices do students (as potential software engineers) need to be trained in?

(Asked by: username_0 on 2017-08-02)

The kinds of things that people with technical training can do have societal impacts, both for good and evil. Programmers are building the future. One future nightmare is [Skynet](https://en.wikipedia.org/wiki/Skynet_(Terminator)).

Many problems in the world have been caused by scientific hubris as well as the *Fallacy of the Last Move*, a concept from Game Theory as it applies to Arms Control and other strategic thinking.

Unbound technologists, especially if amoral, can be very dangerous to the earth and its inhabitants.

So, it seems that future Software Developers need some training in ethical concerns. What, to you are the most important issues and where in the curriculum should they be taught?

Note that ethical issues can be simple or complex. Even something as simple a accepting buggy software can have ethical implications and such lessons can be taught in early courses.

At the other end of the scale is the question of how much a person owes to their employer vs to humanity as a whole.

* What essential ethical practices do students need to be trained in?
* What are effective ways of integrating ethics into the curriculum?
* What are the first steps a teacher should take in integrating ethics into the curriculum?

A general discussion of Computer Ethics is: <https://plato.stanford.edu/entries/ethics-computer/>

**Tags:** curriculum-design, best-practice, ethics

> **Comments:**
> **username_1**: I think faulty engineering is a likelier ethical concern than Skynet.  
> **username_2**: The BCS Code of Conduct might give some insight into this issue; http://www.bcs.org/upload/pdf/conduct.pdf  
> **username_3**: Speaking as a machine learning guy, I *wish* creating AIs was actually easy enough that a rogue programmer creating Skynet was a realistic concern.  
> **username_4**: I highly recommend "Ethics and Technology: Controversies, Questions, and Strategies for Ethical Computing" by Herman T. Tavani.  
> **username_5**: Some good case studies, would be Uber (http://www.bbc.co.uk/news/business-36843386), Ashley Madison (The site itself, and the subsequent information disclosure), Encrypted Messaging vs National Security (http://www.express.co.uk/news/world/834597/WhatsApp-Facebook-removing-encryption-would-make-extremists-harder-track)  
> **username_6**: @nocomprende: as someone that survived lymphoma I really, really disagree with you.  
> **username_6**: @nocomprende: I still believe that the question cannot be stated like that. Lack of food and water are about lack of social structures, such as Democracy, that establish societies where people's wellfare is guaranteed. The cure for diseases is a completely different thing. Of course you could argue that sciencific and technological development are only possible where those basic problems are already solved, but if they are, then we will not have to choose. I think that your dichotomy is not well put.  
> **username_0**: Conversations like the above are appropriate in the classroom. https://chat.stackexchange.com/rooms/59174/the-classroom. Less so in comment streams.  

---

## 9 answers

### Answer by username_13 on 2017-08-02. Score: 15

I'm going to begin by quoting Ken Thompson's Turing Award Lecture "Reflections on Trusting Trust" ([link](https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf)).

> To what extent should one trust a statement that a program is free of
> Trojan horses? Perhaps it is more important to trust the people who
> wrote the software.

In my mind the first and most foundational lesson to impart is this: **software, computers, technology -- these things are more about people than about code.** There is a trust from the end-user, whoever that user may be, that must not be violated. Users are not means to an end; they are the end. The code is the means.

I am reminded of the [Second Formulation of Kant's Categorical Imperative](https://en.wikipedia.org/wiki/Categorical_imperative#The_Second_Formulation:_The_Formula_of_Humanity):

> Act in such a way that you treat humanity, whether in your own person
> or in the person of any other, never merely as a means to an end, but
> always at the same time as an end.

In terms of integrating ethics, the way to start in my mind is discussion. Early and often. Share news stories. Discuss implications of new apps students download. The first 12 minutes of [this CS50 lecture](https://www.youtube.com/watch?v=8Ba7SgCN2XY) cover the the Volkswagen/EPA scandal from 2015 and mention Ken Thompson's lecture. It is essential that awareness is built right from the beginning that software is everywhere and there needs to be an ethical code to guide that development. In my mind, while I wouldn't consider myself Kantian, I do think his point about means and ends is essential.

The AP Computer Science A curriculum outlines the following requirements under the major topics of the course:

> **VI. Computing in Context**
>
> An awareness of the ethical and social implications of computing
> systems is necessary for the study of computer science. These topics
> need not be covered in detail, but should be considered throughout the
> course.
>
> A. System reliability
>
> B. Privacy
>
> C. Legal issues and intellectual property
>
> D. Social and ethical ramifications of computer use

Similarly, in AP CS Principles, students, as a part of the Explore Task, have this on their rubric for the assessment:

> **Analyzing Data and Information**: Identifies one storage, privacy,
> OR security concern. Explains how the concern is related to the
> computing innovation.

The moral of the story here is that ethics must be discussed with students as soon as possible whether it is a high school AP course or an introductory computer science course at a university.

This idea of **trust** must be imparted to students, and it must be emphasized that the world of software engineering and computer science in general is not in a moral vacuum. It does no good to address a particular issue, say data privacy, without first contextualizing it within a larger ethical system, one that sees each user as an end, not a means, and as someone who is placing trust in you the developer.

> **Comments:**
> **username_13**: @nocomprende I don't disagree. I am reminded of the great wisdom of Dr. Ian Malcolm from _Jurassic Park_: "Your scientists were so preoccupied with whether or not they could that they didn't stop to think if they should." People need to "stop to think." Instilling that "need": "Ay, there's the rub."  
> **username_14**: The only person I would trust is a madman who would rather voluntarily live in poverty than allow himself to be pressured. Well, since everyone else by definition can be pressured and bought to sneak in bad stuff in the code which therefore eradicates any trust.  
> **username_14**: @nocomprende no idea what this "Tao" says or who he is. But there do definitely exist enough ways to pressure most people to stop trusting the concept "trust" altogether.  
> **username_0**: @username_14. Even the *Road Warrior* world had *some* trust. You have to get to *Lord of the Flies* to go without it. But perhaps you only mean "trust but verify".  Also, conversations to the classroom, please: https://chat.stackexchange.com/rooms/59174/the-classroom  

---

### Answer by username_15 on 2017-08-03. Score: 4

# None.

Yes, a bold statement, literally. So let me elaborate further.

You cannot cover all potentially relvant aspects of "ethical practices" in a computer science course. Assuming that this is a general, basic course, you simply cannot foresee the potential ethical questions that will arise for these students. One of them may work in the IT department of a hospital. He should not sell patients data to a pharma company. One of them may work in a bank. He should not disclose or manipulate the financial status of customers. Another one will design brain-computer interfaces. He should not use his abilities to alter the behavior of the subject in a non-ethical way (*whatever this means!*). Another one may be responsible for implementing the emergency handling of a self-driving car, and the ethical implications of that are currently the subject of deeply philosophical (and still open) discussions

Related to that is another problem: You can hardly ever foresee the ethical implications that your work may have. Although this example has been used ad nauseam (and may be considered as some sort of a fallacy), I like the comparison of computer science and a sharp knife: You may use it to stab someone, or to perform a life-saving surgery: The data analysis software for weather forecasts may be used to compute questionable credit scorings. The image analysis software that you're implementing to sort holiday photos may be used to identify people in surveillance camera images.

(This has a huge potential for abuse. The fact that people could now argue that this "helps to catch terrorists" shows how difficult this topic is. The ethical discussion *itself* is beyond the scope of this answer and this site. In fact, the examples here should only show *exactly* this: You cannot summarize "ethics" on a few PowerPoint slides...)

And finally, assuming that the students are in their 20's: It's too late. You will not convert an (ethically) "bad" person into an (ethically) "good" person by listing things that they could do but should not do. Acting responsibly (in an ethical sense) is nothing that can (or should) be tought alongside the basics of abstract data types, computer architecture or object oriented design...

---

To summarize, although most of the downvoters will likely not read up to this point: A computer science course is not the place to **train** people in **ethical practices**, as stated in the question.

What you may (or even **may have** to) teach is **awareness**. People should not act unethically (or just aid others in acting unethically) due to a lack of knowledge. Computer scientists, particularly in their later (practical) roles, for example, as IT administrators, have a lot of power, and thus, a lot of *responsibility*. They should be aware of the fact that their knowledge or work (or a moment of carelessness) may literally destroy lifes.

This awareness about the implications of their work is unrelated to ethics. Whether or not they use their powers in an ethical or unethical way is nothing that you can "teach" them.

> **Comments:**
> **username_16**: You seem to be contradicting your bold statement (I noticed the pun). If I understood correctly, you are saying that *because of the serious dangers that unethical IT workers might pose*, they **shouldn't** be taught not to do those things? Even if one cannot "foresee" the ethical implications work might have, I doubt it will ever be considered ethical to disclose secret information which leads to **direct** harm to others. That can be taught, and [You don't need the gift of foresight](http://www.quotes.net/mquote/120399) to know what sort of ethics would be useful for your students.  
> **username_15**: @username_16 The point is that *teaching* (and thereby to some extent *"defining"*) ethics is not sensibly possible in a CS course. Regarding your example: Disclosing information that may harm others could be "OK" if the "other" person is a terrorist who plans an attack. Whether this is unethical can neither sensibly be discussed here nor in a CS course. On the other hand, what you **can** teach is, for example: **Which** measures may prevent unauthorized access to personal data, or which pieces of information can be combined in order to identify someone, from a purely technical point (...cont'd)  
> **username_15**: (also @nocomprende ) : To **judge** whether something is ethical/unethical, right/wrong, good/evil depends on the context. Although you can [define a motto, like Google did](https://en.wikipedia.org/wiki/Don%27t_be_evil), this won't mean anything. Although there certainly is some consensus about what is "right" or "wrong" (regarding WW2, most people will call it "wrong"), **teaching** this is totally unrelated to the contents of basic CS courses. Or another summary: You should not teach *ethics*, but the (technical) points of the work that may have *implications* for ethics.  
> **username_0**: Let me suggest that ethics isn't an infinite set of "rules": if A then B, but a set of principles and dispositions to act in a certain way. Physicians have been trained "Do no Harm" for quite a long time, though the quote is attributed to Florence Nightingale. Too many rules: yes - don't even try. Infinitely applicable principles: yes  
> **username_17**: "... assuming that the students are in their 20's: It's too late. You will not convert an (ethically) "bad" person into an (ethically) "good" person..." This assertion is [not supported by the research](https://www.scu.edu/ethics/ethics-resources/ethical-decision-making/can-ethics-be-taught/) about ethics and morality. We don't even seriously *start* developing a cohesive moral perspective until our 20s, and this process continues well into our 30s.  The very late teens and the early 20s are prime time in the brain for discussing moral and ethical action.  
> **username_15**: @BenI. It's not tooo surprising that the ethics department of a university claims that ethics can be tought ;-) But that's not the point. The point is rather that this ethics department will not teach the students that ethical implications can be modeled with binary logic on NAND gates. One could blatantly say: "That's off-topic". Again: CS students should be aware of their responsibility and how their work may affect the lifes of others. But trying to *teach* them what is "right" or "wrong" (and for which reason) seems pointless for me, as these are subjective, open-ended discussions.  
> **username_15**: @nocomprende The trolley example is *very* obvious, and I referred to it in the answer. There are dozens of examples that are similarly (although not so obviously) subtle. To put it that way: The CS/engineer will not decide about the track that the trolley will take, but he will have to make sure that there is a lever to switch the track in the first place, and be aware that, if this lever does not work, something will happen what *others* defined to be ethically wrong (regardless of what this is).  

---

### Answer by username_11 on 2017-08-03. Score: 4

> What are the first steps a teacher should take in integrating ethics into the curriculum?

I would begin with practical, everyday topics which every student can relate to, and which are (should be) absolutely practically relevant in todays world from day #1 for every developer. Maybe based on current newspaper-level discussions.

* Privacy (being able to stay private when using an application, controlling who can see what you do)
* Data security (being able to protect your own data, e.g. in the context of a business)
* Personal data (being in control of data related to yourself as a human being; related but not equal to Privacy)

You should easily be able to find an abundance of concrete examples, let me give you some that come to mind:

* Privacy: communicating with someone while being sure that nobody can read your chat. May it be your spouse (say when organising an expensive birthday present for them), your system administrator (when executing your function as a manager to discuss your subordinates with your boss, etc.), or the general public (obvious reasons...).
* Data security: Keeping contract offers away from competitors.
* Personal data: Keeping preferences, habits, interests etc. from prospective employers or insurance companies.

I would ask the pupils to come up with their own, they should be able to find a lot of different things.

This, then leads to your question: it is ethically important for everybody involved in the creation of software systems to further these things. I.e., not to be laizzes-faire about them, not to prioritize them lowly when budget or time pressure arises, not to actively violate them; on the other hand, to constantly remind/teach the stakeholders, who may not have an understanding of these matters, and so on.

> **Comments:**
> **username_11**: @nocomprende: absolutely. Not only the Web, but the Internet as a whole. I was active in Fidonet, Usenet etc.; one day DejaNews, then Google, just slurped in the whole Usenet history, for eternity. Before that, Usenet was kind of temporary. That was kind of *the* eye opener for me; before that nobody gave a d*mn about what they posted (private information, etc.); afterwards not so much. If I check Google Groups now, I easily find all my post history from the early 90's, including real names, mail addresses, phone numbers in the signature and such things...).  

---

### Answer by username_18 on 2017-08-03. Score: 3

Raise awareness of the issues, discuss historical (Therac 25 is an old but highly interesting case of software bugs directly causing people to get hurt) and current cases (EU/US safe harbor and followups - there is so much potential to go in-depth about what data protection means and what the risks are!), and maybe make the legal situation (especially international law!) part of the curriculum ... but don't try to "train" people towards any ethical viewpoint. The latter is in the end straight manipulation no matter how well intended, and something people should if at all be trained to guard against.


---

### Answer by username_19 on 2017-08-05. Score: 3

Teach them to be cautious. This may not sound like ethics, but actually it has a significant ethical effect.

People - especially young men - who are very confident try all kinds of things, including unethical ones. Their confidence is often misplaced. Computers will not forgive you because you are young and charming. They'll happily retain the evidence of wrongdoing, and it's very hard to clean it off them completely.

Misuse of computers tends to be discovered. Unlike people, computers do not cover up evidence of deception, because they're incapable of being embarrassed.

Besides, caution is sound engineering practice. You get two useful results from one set of teaching.


---

### Answer by username_7 on 2017-08-27. Score: 1

# Software craftsmanship

As professionals, we need a set of ethics (ethics is not the same as morality). Medics have ethics, even lawyers have ethics. They can be struck off from their profession if they do not stick to them. If we do not create our own, then they will be imposed on us (see talk by [Uncle Bob Martin](https://www.youtube.com/watch?v=ecIWPzGEbFc)).

In some ways it does not matter exactly what they are. You could have a class discussion. You could include those proposed by others. There are many points of view. A few contradict, but many do not.

I have included those proposed by Uncle Bob et al.

## Agile manifesto

We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:

* Individuals and interactions over processes and tools
* Working software over comprehensive documentation
* Customer collaboration over contract negotiation
* Responding to change over following a plan

That is, while there is value in the items on
the right, we value the items on the left more.

## Software craftsmanship manifesto

As aspiring Software Craftsmen we are raising the bar of professional software development by practicing it and helping others learn the craft. Through this work we have come to value:

* Not only working software,
  but also well-crafted software
* Not only responding to change,
  but also steadily adding value
* Not only individuals and interactions,
  but also a community of professionals
* Not only customer collaboration,
  but also productive partnerships

That is, in pursuit of the items on the left we have found the items on the right to be indispensable.


---

### Answer by username_7 on 2017-10-27. Score: 1

I was recently watching videos of [Libra Planet 2017](https://libreplanet.org/2017/). A lot of their talks covered ethics.

There seems to be a **lot** of ethical considerations in software. They did not cover everything, but here is a summary of some of the topics:

* Jobs.
* Security.
* Privacy.
* Equal rights / user Disability.
* User rights: Right to use, right to study, right to distribute, right to change …
* Algorithmic bias: e.g. search, selection (for jobs), crime detection.
* Law: The good, the bad, and the ugly.
* Politics.
* Free speech: Software that you can go to jail for talking about (in the USA).
* Workers(programmers) rights and responsibilities (guilds and trade bodies).

Some things not covered.

* Safety.
* Artificial Intelligence.


---

### Answer by username_7 on 2017-08-02. Score: 0

# The additional dangers of software over other engineering fields.

## Who is in charge

All software is a form of artificial intelligence, by this I do not mean artificial consciousness (e.g. skynet, I Robot, ExMachina, robby, and every other robot from the movies).

I think that the probability of artificial consciousness arising soon is very low. However some form of artificial intelligence is in every bit of software. The danger is not the artificial intelligence, but who controls it. A lot of software to day is controlled by big corporations. Users of the software are not free to: choose how they use it; study how it works; make changes to it; share it with their friends.

Therefore all software should be Free Software.

I am not referring to Open Source. Although (Software that is Free Software) ≈ (Software that is Open Source). I am referring to the concept of Free Software. The freedoms, for it is the freedoms that we need. These freedoms are also in the Open Source definition, but are usually over looked.

I am also not saying that all Free Software is ethical. There are many sides to being ethical. There may also be proprietary software that is more ethical than a particular bit of Free Software. **Free Software is necessary but insufficient, to being ethical.** Just as obeying one of the 10 commandments is necessary but insufficient, to being good (Even obeying all of the law is insufficient).

> “All these [laws] I have kept,” the young man said. “What do I still lack?”.
> He answered, “If you want to be perfect, go, …”

What I am saying is about the ethicalness of the software, not of the user.

Free Software has nothing to say about money (except to say that you must be free to charge money for it).

## Artificial consciousness

Just a small paragraph to counter what I said above.

If some one interfaces a biological brain to a computer, and gives it power. That is the ability to do harm, then it probably will. Any disembodied brain will have no empathy.

> **Comments:**
> **username_8**: So the only ethical software is "free" software, where "free" is defined by Stallman's turgid GPL. I disagree.  
> **username_7**: @JamesKPolk but not all Free Software is ethical. I have amended answer to address this. You are saying that because software A has some aspect of ethical then it **is** ethical. And You are thinking that I am saying that all Free software is ethical. It is not that simple.  
> **username_7**: @JamesKPolk The Free Software Definition defines what Free Software is. The GPL is but one implementation. You should give them a read, you will find that the language is very simple and short.  
> **username_9**: -1 if I could downvote on this site. This doesn't seem to have anything to do with ethical software.  
> **username_10**: Why would a disembodied brain have no empathy? Are you suggesting empathy comes from the liver or what? It's not a separate system that restricts the brain, it's part of the brain. Not that I'm saying it's a good idea trying to model a true artificial intelligence after the ridiculously flawed machine built by natural selection :)  
> **username_7**: @username_10 No I was thinking that empathy comes from your interaction with others. But I would leave it as a discussion for the students, to decide if they think I am correct.  
> **username_7**: @nocomprende https://www.youtube.com/watch?v=IHyH3MPgZDo  

---

### Answer by username_7 on 2017-08-02. Score: -1

There are certain unsolvable problems (I am not referring to NP), that seem solvable, and an apparent solution can be made and deployed. These systems can be used, but do not fulfil there requirements (security, privacy, …).

This differs from the case where the software lacks these features, because of poor workmanship (also an ethical issue), in this case it can not be done. I turn down the work, because it is impossible. The client thinks they need to find someone that is more clever, that can do it. Eventually they find someone that has no idea of the problems, that agrees to do the project. If the new supplier realises that it can not be done, and tells the client, then the process repeats, else a faulty product is delivered.

An example is electronic voting.

It is impossible to create an electronic voting system that meets all of the requirements. If we just have 2 of the requirements for a voting system.

* Anonymity
* Trustable

Then we can not produce an electronic system, that can meet just these requirements. There are also other requirements, thus it is even harder (in the higher than infinity sort of way (∞+1 = ∞), therefore we don't need to look at the other requirements, to see how hard it is).

How would you audit this system? You could check its source code, to see that it does the right thing (if you can read it, so every one involved would have to have a good enough level of CS, and know the language). Oh but how do we know that the system is running this code.

Every single solution to this problem, has a counter measure ( a dis-proof ).

This has not stopped these things from being deployed many times, and in countries that are considered to have a well run voting system.

---

# What is needed

A box, that you put all of your votes into, and out comes a count for each candidate.

# Possible solutions

How do you know that the output is correct. Ah that is easy, you produce an audit trail. But now we have no anonymity. So we add encryption. But for it to be audit-able, anonymity can still be broken. (If you can check your own vote, then I can check your vote (if I torture you long enough, then you will give up your key).

# A solution that works, but does not use a computer.

The many boxes
- you walk into a room (box1), and show your ID, your name is removed from the list, and you are given a voting sheet.
- you walk into a subroom (box2), and make your mark on the sheet.
- you then fold(box3) the sheet.
- you leave box2 and walk to a locked box (box4), and place your sheet into it.
- at all times observers (from all interested parties) are walking around box1, to check that the rules are being followed.
- at the end box 4 is sealed.
- latter box 4 is opened and the sheets counted. With lots of people watching. A set of totals is produced and announced. The sheets are put back in box 4, sealed and locked.

I have probably missed some detail, but this solution cannot be implemented electrically, because there is no way of putting people into the electronic boxes to observe.

# Some videos on this

[The problem](https://www.youtube.com/watch?v=w3_0x6oaDmI)

[Some solutions](https://www.youtube.com/watch?v=BYRTvoZ3Rho) I found a problem with all of these solution, as I was watching it (Did not have to sit and think for days).

[Other stuff](https://www.youtube.com/watch?v=ZM-i8t4pMK0)

> **Comments:**
> **username_9**: You make it sound like paper voting somehow has less issues with trustworthiness. Technical solutions could theoretically improve trustworthiness (as you increase the number of people who would all need to be 'in on it'), but it's a very complex issue. Anyway, -1 if I could vote down on this site.  
> **username_7**: @DavidMulder So if I provide a system, for an election. It checks each voters ID. We could ensure anonymity by never ever ever, connecting it to a network after it has been used. However there is no way for the voters to know. Then it takes your vote, and reduces it to a count for each candidate. How will you as a candidate ensure that it does this correctly ( It could just make up the numbers, in favour of the candidate that it wants to win).  
> **username_7**: @nocomprende it has already been solved for paper based system, but you may have improved it. However I see no need to use dollar bills: once country elects an untrustworthy precedent, the dollar bill would be as trust worthy as raffle tickets. But still an improvement, as it allows each voter to verify there part. The existing system focuses on the candidates and parties verifying the system. Can you see why it is impossible to implement using a computer system?  
> **username_11**: While the answer has some merit in the context of the question (for example as a teaching aid to show layers of complexity for the use case of electronic voting systems), I would be *really* reluctant to make the  absolute claim that it is prinicipally impossible. Cryptography gives us tools similar to the pen&paper, anonymous envelops, tokens, booths...; without having really analyzed it, I could see possible solutions. Obviously, part of it would be that local machines (+ their software) would be assumed to be compromised, and such compromising must be detected by the "crypto" part.  
> **username_7**: @username_11 because you can not trust or audit the computer, you need end to end checking (crypto). Now get this to work with anonymity and inability to show how you voted. The some solutions video talks about these, but I see problems with all of the solutions (and I am to that clever). This is without having to resort to “How does my mum verify a cryptographic hash by hand (because she dose not trust the computer).”  
> **username_11**: @richard: my point is not that I have outlined a working electronic voting system in a SE comment. ;-) My point is that is incredible hard to argue that "solving XYZ is not possible and will never be possible". It is absolutely fine to argue that todays solution to XYZ suffer problems A, B, C and that nobody has yet found a working solution, but your answer seems to say that it is theoretically and in all principles impossible to solve the problem.  
> **username_7**: @username_11 My point is that no implementation can ever be found, because the way it is implemented is impossible. You can be a black box analysis of the requirements. The requirements contradict: Actually the real requirements are very hard, and only just possible, but if you say that you must have a computer, you add an implied requirement that no one can observe. With this no requirement there becomes a contradiction.  
> **username_7**: @username_11  I just looked up homomorphic encryption, just to see what it was, because there is a clam that this solves the E-voting problem. I predicted that it must have the property of “whoever can read the result, can find out who you voted for”, for it to do what it was claimed to do. I don't understand the math of the fully homomorphic systems, but their usage manuals confirmed my fear. It does not matter what happens in the box, it must have these properties, therefore in is of no use in implementing an E-Voting system. It looks like it may have other uses though.  
> **username_7**: Since my lost comment on this, I heard a talk by a women that has her Phd + in electronic voting systems, she has several **mathematical proofs** that they can not do what they need to do. One of her colleges invited a group of “young people” to hack some real voting machines (manufacturers claimed 100% secure, and trustworthy) , in a 1 or 2 week session. She was sceptical that there would be time, to find any thing. First exploit found in two minutes. All compromised by end of exercise. Could be that all current systems, are made by cowboys. But see proofs, or linked video for general case.  
> **username_12**: I'm not seeing why this fits under "*ethical practices*".  
> **username_7**: @username_12 Is it ethical to have the people use voting systems, that do not do what they say they do. That do not do what the voting laws say they must do.  

---

