Cryptocurrency analysis and predictions using AI and big data

Discussion in 'CryptoCurrency' started by healzer, Jan 3, 2018.

  1. healzer

    healzer Jr. VIP Jr. VIP

    Joined:
    Jun 26, 2011
    Messages:
    2,951
    Likes Received:
    2,936
    Gender:
    Male
    Location:
    Somewhere in Europe
    Thanks for the feedback :D
    The domain has been carefully chosen (smart marketing tactics haha).

    1) I have temporarily disabled the horizontal lines as you mentioned, I'll add an option to toggle it on/off soon.
    2) The thing is that the price is coupled to the left axis, while all other data (which is relative) is coupled to the right axis. So what you are doing on that GIF is moving the price-axis, but both are completely independent. If we don't do this then the price will be super high (e.g. range of ±10k) while all the other data will be between -100 and 100 and so you wouldn't be able to see anything clearly.

    I hope this makes some sense :)
    Have a nice weekend.
     
    • Thanks Thanks x 1
  2. Yes, I understand that, but is there a way to move both of them proportionally?
     
  3. healzer

    healzer Jr. VIP Jr. VIP

    Joined:
    Jun 26, 2011
    Messages:
    2,951
    Likes Received:
    2,936
    Gender:
    Male
    Location:
    Somewhere in Europe
    Well it is always possible to find a way to hack around it.
    However, from a practical point of view, how important/necessary is it to move them around? Because all data is clearly visible within in the graph's window, including the drops/rises and trends. Thanks!
     
  4. Bel1616

    Bel1616 Jr. VIP Jr. VIP

    Joined:
    Jan 6, 2017
    Messages:
    244
    Likes Received:
    52
    Gender:
    Male
    Amazing man! I was waiting this, Really good job, keep it Up!
     
    • Thanks Thanks x 2
  5. healzer

    healzer Jr. VIP Jr. VIP

    Joined:
    Jun 26, 2011
    Messages:
    2,951
    Likes Received:
    2,936
    Gender:
    Male
    Location:
    Somewhere in Europe
    Hi guys :)

    I have got something exciting for you.
    Today and yesterday I have invested all my time implementing the auto-prediction system.
    This new feature automatically generates predictions for every coin.
    Right now it makes 4 predictions per coin, for 4 different intervals: 30min, 60min, 12hrs and 24hrs.
    Basically It predicts 20 intervals into the future depending on the interval you choose.

    Here are a few screenshots before I go into more details.

    IOTA, 30min intervals:
    [​IMG]

    BTC: 30min, 12hr and 24hr intervals:
    [​IMG]

    [​IMG]

    [​IMG]

    A few technical notes:
    Every 30 minutes
    the system generates a prediction per crypto and per interval.
    There is a reason for generating a prediction for each possible interval: remember that we noticed that the predictions are only accurate for about 5 - 12 intervals into the future. So for instance if we would want to predict 20 days ahead, we don't want to use 30min intervals to generate 960 prediction-points. So instead we train the model on the 24hr-based data and make 20 prediction-points from it.

    Feel free to use the tool here: http://cryptanal.com/predictionsChart.php
    When you are going to use the prediction tool, you may see an empty chart -- this is because it has not generated a prediction (yet) for the datetime you selected.
    Simply click "prev" and/or wait until the new prediction is made.

    You can use the "prev"/"next" button to go to the previous/next interval. I added the prev/next buttons to see which predictions were made in the past, so we can compare history against reality. However, only a limited number of predictions have been recorded until now (only the most recent ones), so you will not find a prediction made e.g. 10 days ago.

    On the gif below you can see how the prediction evolves when pressing "next" (I started at some historic datetime):
    [​IMG]
    Notice that each prediction looks (slightly) different from the previous one.
    This is because the neural network is re-adjusted with the new data (continuous learning) every interval.

    In one of my earlier posts I explained that there are a bunch of parameters/configurations for the prediction algorithm.
    I have not yet figured out which parameters are the "optimal" ones, this is not a trivial task but it will be done in the near future.
    So the current algorithm uses the parameters which gave me the best results thus far:
    • Sequence length: 20 intervals
    • Features: 5 (price, volume24, social hype, sentiments from news articles, sentiments from social media)
    • Neurons: 2
    • Epoch: 500
    • batch size: 20
    Regarding the price and volume: I may not have mentioned this before, but the price we record is the average price of a cryptocurrency across all mainstream exchanges. The same is true for the traded volume, the system takes the average of the traded volume across all exchanges. Maybe in the future I will implement per-exchange charts which is useful to very serious traders.

    Apart from implementing the prediction tool, I have patched a few bugs.
    The most important one is displaying more digits after the decimal point/comma. Previously it was rounded to 2 digits, but I noticed that some coins are worth less than one US Dollar (e.g. NXT):
    [​IMG]

    Thanks for reading, I love all of your support and feedback!!! :)
    Would love to hear your thoughts on this one as well.
    - Ilya
     
    • Thanks Thanks x 4
    Last edited: Jan 27, 2018
  6. You should also look at time of day & day of the week.

    Some coins are more active on certain time zones and go down on specific days like weekends.
     
    • Thanks Thanks x 2
  7. healzer

    healzer Jr. VIP Jr. VIP

    Joined:
    Jun 26, 2011
    Messages:
    2,951
    Likes Received:
    2,936
    Gender:
    Male
    Location:
    Somewhere in Europe
    Thanks :)
    That is very true. Luckily the neural network takes these parameters into account implicitly as it learns when volume/price goes up or down.
    I'm not sure whether feeding the day of week and/or time of day into the neural net would yield additional value, but definitely something worth trying out.
     
    • Thanks Thanks x 1
  8. mindmaster

    mindmaster Jr. VIP Jr. VIP

    Joined:
    Sep 16, 2010
    Messages:
    3,832
    Likes Received:
    1,774
    Home Page:
    Any way of making prediction for ICOs, as well ?
     
  9. healzer

    healzer Jr. VIP Jr. VIP

    Joined:
    Jun 26, 2011
    Messages:
    2,951
    Likes Received:
    2,936
    Gender:
    Male
    Location:
    Somewhere in Europe
    ICOs are not yet supported and will require different analyses. :)
    I may look into that very soon.
     
    • Thanks Thanks x 1
  10. I am also looking at automating ICOs. Really hard (pretty much impossible) to predict sucess automatically.

    I know that the Cindicator bot offers ICO rating, and you could maybe use ICOBench's API but it's honestly just easier to spend an hour or so researching each ICO yourself.
     
    • Thanks Thanks x 1
  11. healzer

    healzer Jr. VIP Jr. VIP

    Joined:
    Jun 26, 2011
    Messages:
    2,951
    Likes Received:
    2,936
    Gender:
    Male
    Location:
    Somewhere in Europe
    Awesome, thanks for sharing these sources.
    I'll definitely try to use them. :) Cheers!
     
  12. Cindicator bot requires xx,xxx-xxx,xxx holdings of their coin, so it's not easy to see what it does exactly.

    You may find this video interesting. The guy in it is a bit weird, but great info.

     
    • Thanks Thanks x 1
  13. Bel1616

    Bel1616 Jr. VIP Jr. VIP

    Joined:
    Jan 6, 2017
    Messages:
    244
    Likes Received:
    52
    Gender:
    Male
    Hey as always good job! I'm a big fan of this project, will you do cryptocurrency only or add on the big stocks in the future?
     
    • Thanks Thanks x 1
  14. Dave Dubauskas

    Dave Dubauskas Newbie

    Joined:
    Jan 28, 2018
    Messages:
    3
    Likes Received:
    3
    WOW! Well done!
    I have not read all the posts so please disregard this suggestion if it has already been posted. If I could suggest it would be useful to included social indicators for trading. What do I mean? Well, for instance, there is a bot that tracks "mentions" of crypto coins on twitter. Google trends also indicate interest in buy or selling coins. I read (somewhere/sometime) ago an article that listed and discussed the "social indicators" of trading... but of course, I can't find it now to post here?! You may wish to keep it separate at first and study the correlations to the actuals? You will decide of course whether there is merit in including some the indicators in your price predictions. Again well done and thank you for work. Cheers.
     
    • Thanks Thanks x 1
  15. st.bandz

    st.bandz Newbie

    Joined:
    Jan 1, 2018
    Messages:
    3
    Likes Received:
    1
    Gender:
    Male
    This is very useful. I'm looking to invest soon.
     
    • Thanks Thanks x 1
  16. healzer

    healzer Jr. VIP Jr. VIP

    Joined:
    Jun 26, 2011
    Messages:
    2,951
    Likes Received:
    2,936
    Gender:
    Male
    Location:
    Somewhere in Europe
    Hey, thanks for the feedback :)
    If I understand your correctly, then we already have that implemented. It is actually our entire backbone.
    Per crypto we track the mentions (aka hype) on Twitter, Facebook and Reddit (these are the social channels).
    But we also track them on big news sites (cnn, cnbc, bbc, ... and 30+ others).
    Soon I would like to expand to blogs and forums as well.
    Hope this answers your question?

    Thanks man, I love your support!
    I had this conversation with a few people some time ago.
    It would be really great if we could expand to the traditional stock market as well -- that's more of a long-term goal and has a lot of its own challenges.

    Cheers!
     
    • Thanks Thanks x 1
  17. ses

    ses BANNED BANNED

    Joined:
    Oct 28, 2017
    Messages:
    48
    Likes Received:
    20
    Based on the last gif where you show how your predictions evolve and the real stats proceed over time it seems like they are not even close to be accurate.

    I am wondering if flipping a coin for each interval with the options for 'up' and 'down' would result in more accurate predictions.
     
    • Thanks Thanks x 1
  18. olm75

    olm75 Senior Member

    Joined:
    Jan 14, 2009
    Messages:
    895
    Likes Received:
    102
    Healzer,
    This is a great thread and project, I'm just sorry I got in so late.
    I day trade Crypto and current use scanners to find coins that are going up but I only scalp 2 - 5% or more if I know for sure how much it will go up in a very very very short period of time using the 5min Candles.
    I have been planning to Diversify a little more:
    DayTrade - 70%
    HODL - 15%
    ICO - 15%
     
    • Thanks Thanks x 1
  19. healzer

    healzer Jr. VIP Jr. VIP

    Joined:
    Jun 26, 2011
    Messages:
    2,951
    Likes Received:
    2,936
    Gender:
    Male
    Location:
    Somewhere in Europe
    Great question!
    On first sight it may appear to be inaccurate, but there is a reason for that.
    Allow me to explain how it works.

    Here is the gif @ses is referring to:
    [​IMG]

    A certain frame on this gif is the chart below.
    It was generated for the interval 22:00 --> 22:30 (GMT+1).
    And it shows that the price is most likely to crash/go down from 11.5k to 10.9k:

    [​IMG]

    But after a few hours its outcome has been adjusted.
    It learned that the price didn't crash so the outcome started to change:

    [​IMG]

    But if we look on the screenshot below, and compare it with the prediction above from 06:00 to 12:00 , it appears to be a pretty good prediction of how the price evolved. At first it went up a little in the first few hours and then slowly started going down. But right now (as I am writing this post), it indicates the price may crash/go down again:

    [​IMG]

    *edit* Here's an easier way to look at what I mean:
    [​IMG]

    It's important to remember that the screenshots above all use 30min intervals. So the neural network is only trained on plots at 30min intervals, it's not very good at predicting e.g. +8 hours into the future because that's already 16 intervals it has to predict, while we noticed it's only good for up to 5-10 intervals into the future.
    So to tackle this problem you may want to use hourly/12h/daily intervals instead:


    Below are two screenshots which use hourly intervals.
    The first one makes a prediction on Jan 27, 22:00 for the next 20 hours:
    [​IMG]

    The second one I have pulled from the site right now and shows how the price evolved, including its prediction:
    [​IMG]

    Now we can compare the first prediction with the 2nd screenshot's actual data.
    Now find the x-tick "Jan 27, 22:00" on both screenshots and look how the prediction differs from actual data.
    We do see that at 02:00 --> 06:00 the prediction showed it would go down, the reality was less worse than indicated, we did have a dip at 01:00 and at 04:00.
    Then from 06:00 until 14:00 the price went up and both areas look very similar, so the prediction is pretty accurate for this zone.
    Here is a better illustration of what I mean:
    [​IMG]

    And finally, for the 12hr interval we have the following (see image below).
    It predicted the price to go up to 11.5k and so it did in reality.
    [​IMG]

    On first sight it appears that hourly/12h intervals are much more accurate than 30min intervals.
    We don't have enough data yet to analyze the 24hr intervals, but in due time we'll dig deeper.

    I hope this clears things up a bit :)
     
    • Thanks Thanks x 2
    Last edited: Jan 28, 2018
  20. ses

    ses BANNED BANNED

    Joined:
    Oct 28, 2017
    Messages:
    48
    Likes Received:
    20
    I exported a report for the next several days in the 12h interval and will check how accurate it is.
     
    Last edited by a moderator: Feb 1, 2018