<p>see eg <a href="http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=785521" rel="nofollow">Neural network training with constrained integer weights</a>
Plagianakos, V.P.;   Vrahatis, M.N.;  </p>

<p><a href="http://www.dice.ucl.ac.be/Proceedings/esann/esannpdf/es1999-18.pdf" rel="nofollow">An Integer Recurrent Artificial Neural Network
for Classifying Feature Vectors</a> 
Roelof K Brouwer PEng, Ph</p>

<p>the general idea in various implementations is to represent/approximate real numbers $x \in[0..1]$ as integers in the form $x \cdot 10^m$ where $m$ is some integer exponent.</p>

CC BY-SA 3.0
2012-01-25T15:53:25.593Z
2012-01-25T15:53:25.593Z
vzn
<p>Most implementations of stochastic gradient descent (SGD) rely on floating points.
Is there implementations using infinite or finite precision integer arithmetics ?</p>

CC BY-SA 3.0
2012-01-21T04:32:12.373Z
Is this question about the theory of stochastic gradient descent or about finding a suitable software package?
Lev Reyzin
CC BY-SA 3.0
2012-01-21T17:34:29.600Z
The former. I'm not looking for a software library for SGD but rather if there is a paper describing a variant of the algorithm using integer arithmetics.
Ghassen Hamrouni
CC BY-SA 3.0
2012-01-25T15:39:48.627Z
why would you want to do that? performance? limited hardware?
vzn
CC BY-SA 3.0
2012-01-25T16:26:36.133Z
I don't really understand the question.  Can't we just represent the floating points using integers?
Lev Reyzin
CC BY-SA 3.0
2012-01-25T18:09:41.067Z
eh - representing floating point via ints causes all kinds of numerical problems. The "correct" approach is to use adaptively growing (or even infinite) precision as is done in many geometric questions.
Suresh Venkat
CC BY-SA 3.0
2012-01-25T19:31:01.697Z
I see. This reminds me why I never went into numerical analysis...
Lev Reyzin
CC BY-SA 3.0
2012-01-28T22:53:52.413Z
I am not sure I understand the point of your question. Why not just change the algorithm to use other representations? (like nested intervals?)
Kaveh
<optimization><machine-learning>
Stochastic Gradient Descent with integer arithmetics
Ghassen Hamrouni
# Stochastic Gradient Descent with integer arithmetics

(Asked by: username_0 on 2012-01-20)

Most implementations of stochastic gradient descent (SGD) rely on floating points.
Is there implementations using infinite or finite precision integer arithmetics ?

**Tags:** optimization, machine-learning
> **Comments:**
> **username_1**: Is this question about the theory of stochastic gradient descent or about finding a suitable software package?  
> **username_0**: The former. I'm not looking for a software library for SGD but rather if there is a paper describing a variant of the algorithm using integer arithmetics.  
> **username_2**: why would you want to do that? performance? limited hardware?  
> **username_1**: I don't really understand the question.  Can't we just represent the floating points using integers?  
> **username_3**: eh - representing floating point via ints causes all kinds of numerical problems. The "correct" approach is to use adaptively growing (or even infinite) precision as is done in many geometric questions.  
> **username_1**: I see. This reminds me why I never went into numerical analysis...  
> **username_4**: I am not sure I understand the point of your question. Why not just change the algorithm to use other representations? (like nested intervals?)  

---

## 1 answers

### Answer by username_2 on 2012-01-25. Score: 3 | Accepted answer

see eg [Neural network training with constrained integer weights](http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=785521)
Plagianakos, V.P.; Vrahatis, M.N.;

[An Integer Recurrent Artificial Neural Network
for Classifying Feature Vectors](http://www.dice.ucl.ac.be/Proceedings/esann/esannpdf/es1999-18.pdf)
Roelof K Brouwer PEng, Ph

the general idea in various implementations is to represent/approximate real numbers $x \in[0..1]$ as integers in the form $x \cdot 10^m$ where $m$ is some integer exponent.


---

