<p>Java has a Streams API that works with all of its collections.  You create a stream with a source, transform or filter it with intermediate operations, and do something with it using a Consumer. </p>

<p>All Collections in Java have a .stream() method that creates a stream from the collection.  You can also create streams from files, as well as creating streams of random objects from a Random.  </p>

<p>Intermediate operations include filter, which accepts a Predicate as an argument, and map and friends that can transform objects into objects of other types or primitives.  </p>

<p>Consumers include facilities to place the contents of a Stream into any Collection, System.out::println, and other pure side-effect operations.</p>

<p>This requires an understanding of lambdas and functional interfaces (our second-trimester course in Java covers these).  This mechanism is powerful and succinct, and it does not alter the collection the stream is attached to.</p>

CC BY-SA 4.0
2019-04-05T14:15:41.503Z
2019-04-05T14:15:41.503Z
ncmathsadist
<p>I'm teaching a theory-focused randomized algorithms course at the undergraduate level, and have included a section on streaming algorithms. It would be great to get students to implement these algorithms, but I do not know how to simulate a data stream for them. I am aware that some companies have APIs with varying degrees of openness (e.g. Twitter), and I'm aware of big things like <a href="https://www.ibm.com/developerworks/library/bd-streamsintro/index.html" rel="nofollow noreferrer">IBM InfoSphere Streams</a>, but this seems too hard, since the course is mostly focused on theory. Ideally, I'd like something that continuously produces a stream of numbers or words, and have students write Python code to access the stream, then compute on the sample they take. Are there any freely available tools of this sort? Even, like, a free toy version of IBM InfoSphere Streams?</p>

CC BY-SA 4.0
2018-09-19T18:43:58.390Z
Welcome back! Do you just want a stream of random numbers? If so, I can build an answer off of something like [this](https://crypto.stackexchange.com/a/5543). If not, what data characteristics would be helpful?
Ben I.
CC BY-SA 4.0
2018-09-19T20:06:07.923Z
@BenI., that sounds like overkill. Why aren't pseudo random numbers enough if all you really want is to illustrate the theory.
Buffy
CC BY-SA 4.0
2018-09-19T21:19:14.307Z
Pseudo random numbers are perfectly fine. I just want them to be coming in a continuous stream, rather than stored on the machine. I think InfoSphere stream is basically an environment you program in, that handles the continuous streams of data in the background. So, you would write code that would start "listening" to the stream (i.e. take a sample) and then compute on what you get during that sample.
David White
CC BY-SA 4.0
2018-09-19T23:04:46.700Z
Forgive my ignorance, but I would like to see a description added to the question of the types of operations that the stream processing would perform, especially when you mentioned "stream of words" rather than only numbers. Maybe there is a James Joyce emulator out there. Stream of consciousness, anyone? You could do an Eliza analysis on the stream...
Scott Rowe
CC BY-SA 4.0
2018-09-20T10:23:37.917Z
@Buffy If it is easy to use, why would it be overkill? The back-end of it hardly matters, just how easy it is to connect to, no?
Ben I.
CC BY-SA 4.0
2018-09-20T11:30:21.470Z
@BenI. No reason if it is easy, but it didn't seem that way with a quick look.
Buffy
CC BY-SA 4.0
2018-09-20T15:10:37.543Z
I think I'll just create a giant txt file of numbers (or words, if we want to do a streaming algorithm on word data) and then have the students read it in line by line. To get practice with multiple streams, I can give them several different txt documents. That way, we don't actually need to interface with the internet or rely on an external API, both of which always take resources to learn.
David White
CC BY-SA 4.0
2018-09-20T15:45:28.493Z
A source of randomness is `/dev/urandom`.
ctrl-alt-delor
CC BY-SA 4.0
2018-09-23T22:49:23.360Z
Having a file of data or using a PRNG which can re-run the same sequence would improve testability.
Scott Rowe
<lesson-ideas><resource-request><python>
How to give students practice with streaming computation?
# How to give students practice with streaming computation?

(Asked by: username_0 on 2018-09-19)

I'm teaching a theory-focused randomized algorithms course at the undergraduate level, and have included a section on streaming algorithms. It would be great to get students to implement these algorithms, but I do not know how to simulate a data stream for them. I am aware that some companies have APIs with varying degrees of openness (e.g. Twitter), and I'm aware of big things like [IBM InfoSphere Streams](https://www.ibm.com/developerworks/library/bd-streamsintro/index.html), but this seems too hard, since the course is mostly focused on theory. Ideally, I'd like something that continuously produces a stream of numbers or words, and have students write Python code to access the stream, then compute on the sample they take. Are there any freely available tools of this sort? Even, like, a free toy version of IBM InfoSphere Streams?

**Tags:** lesson-ideas, resource-request, python

> **Comments:**
> **username_1**: Welcome back! Do you just want a stream of random numbers? If so, I can build an answer off of something like [this](https://crypto.stackexchange.com/a/5543). If not, what data characteristics would be helpful?  
> **username_2**: @BenI., that sounds like overkill. Why aren't pseudo random numbers enough if all you really want is to illustrate the theory.  
> **username_0**: Pseudo random numbers are perfectly fine. I just want them to be coming in a continuous stream, rather than stored on the machine. I think InfoSphere stream is basically an environment you program in, that handles the continuous streams of data in the background. So, you would write code that would start "listening" to the stream (i.e. take a sample) and then compute on what you get during that sample.  
> **username_3**: Forgive my ignorance, but I would like to see a description added to the question of the types of operations that the stream processing would perform, especially when you mentioned "stream of words" rather than only numbers. Maybe there is a James Joyce emulator out there. Stream of consciousness, anyone? You could do an Eliza analysis on the stream...  
> **username_1**: @username_2 If it is easy to use, why would it be overkill? The back-end of it hardly matters, just how easy it is to connect to, no?  
> **username_2**: @BenI. No reason if it is easy, but it didn't seem that way with a quick look.  
> **username_0**: I think I'll just create a giant txt file of numbers (or words, if we want to do a streaming algorithm on word data) and then have the students read it in line by line. To get practice with multiple streams, I can give them several different txt documents. That way, we don't actually need to interface with the internet or rely on an external API, both of which always take resources to learn.  
> **username_4**: A source of randomness is `/dev/urandom`.  
> **username_3**: Having a file of data or using a PRNG which can re-run the same sequence would improve testability.  

---

## 1 answers

### Answer by username_5 on 2019-04-05. Score: 3

Java has a Streams API that works with all of its collections. You create a stream with a source, transform or filter it with intermediate operations, and do something with it using a Consumer.

All Collections in Java have a .stream() method that creates a stream from the collection. You can also create streams from files, as well as creating streams of random objects from a Random.

Intermediate operations include filter, which accepts a Predicate as an argument, and map and friends that can transform objects into objects of other types or primitives.

Consumers include facilities to place the contents of a Stream into any Collection, System.out::println, and other pure side-effect operations.

This requires an understanding of lambdas and functional interfaces (our second-trimester course in Java covers these). This mechanism is powerful and succinct, and it does not alter the collection the stream is attached to.


---

