Ignite Twitter Streamer module consumes tweets from Twitter and feeds the transformed key-value pairs <tweetId, text> into an Ignite cache.
To stream data from Twitter into an Ignite cache, you need to:
- Import Ignite Twitter Module in Maven Project
If you are using Maven to manage dependencies of your project, you can add the Twitter module
dependency like this :
<dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-twitter</artifactId>
<version>${ignite.version}</version>
</dependency>Replace '${ignite.version}' with the actual Ignite version you are interested in.
- In your code, set the necessary parameters and start the streamer, like so:
IgniteDataStreamer dataStreamer = ignite.dataStreamer("myCache");
dataStreamer.allowOverwrite(true);
dataStreamer.autoFlushFrequency(10);
OAuthSettings oAuthSettings = new OAuthSettings("setting1", "setting2", "setting3", "setting4");
TwitterStreamer<Integer, String> streamer = new TwitterStreamer<>(oAuthSettings);
streamer.setIgnite(ignite);
streamer.setStreamer(dataStreamer);
Map<String, String> params = new HashMap<>();
params.put("track", "apache, twitter");
params.put("follow", "3004445758");
streamer.setApiParams(params);// Twitter Streaming API params.
streamer.setEndpointUrl(endpointUrl);// Twitter streaming API endpoint.
streamer.setThreadsCount(8);
streamer.start();Refer to Twitter streaming API documentation for more information on various streaming parameters.