- Sep 25, 2013
- 4,705
- 2,701
I'm impressed by the amount of knowledge you shared here, thanks for the read @splishsplash
Do you think Python will be the most used programming language in the future and today?
- Thread starter proxysolid1
- Start date
D
Deleted member 1701249
Guest
I dont think so since python is slow and i dont hope so honestly .
D
Deleted member 1701249
Guest
Python is for creating large applications but as a language its slowComplete opposite
That's the very reason we use languages like Python. It's easier and faster to develop massive applications with less bugs and it's far easier to create an extensive test suite.
It's brutally difficult to do this in low level languages. You're dealing with directly with memory in C++. At that level it's super easy to create bugs that you'll almost never find and will only trigger under rare scenarios. The simplest of tasks in C++ is an opportunity to introduce bugs.
Hence this is what makes it so strange to say C++ is a real language, and Python is just a scripting language that you can't develop large applications in. Python is FOR creating large applications.
- Oct 9, 2013
- 3,471
- 14,452
It's not slow.
No one creates CPU bound programs in Python, and there's zero advantage to creating applications in a language like C++ where it's Network/File IO bound.
All the libraries you need with multiple implementations of pretty much any algorithm in existence can be found as an optimized Python lib which has the CPU bound parts written in C/assembly.
Python wouldn't be used by major companies if it was a slow language.
If you're going to implement bubble sort in pure Python, then yes, it's slow.
But that's like renting a removal truck, then taking it to the race track. It's not what it's meant for.
And in practice, it's not slow, because you use optimized libs.
Look :-
import numpy as np
import pandas as pd
import time
s_t = time.time()
array = np.random.randint(1, 101, size=(20000000, 5))
df = pd.DataFrame(array)
df_multiplied = df * 3
df_multiplied['mean'] = df_multiplied.mean(axis=1)
e_t = time.time()
t_t = e_t - s_t
print(df_multiplied.head())
print(f"Time: {t_t}")
This builds a vector with 5 columns and 20 million rows, multiplies every element by 3 then adds a 6th column "mean" and calculates the mean of each row.
(tom@localhost)-(jobs:0)-(~/projects/tests)
(! 2014)-> python test3.py
0 1 2 3 4 mean
0 189 222 273 39 159 176.4
1 60 153 36 156 24 85.8
2 186 159 129 213 99 157.2
3 288 84 138 279 54 168.6
4 27 243 3 36 21 66.0
Time: 5.617027997970581
(projects-env)
Takes 12.3 seconds to run on my $24/mo VPS on linode
(tom@localhost)-(jobs:0)-(~/projects/tests)
(! 2006)-> cat /proc/cpuinfo | grep name
model name : AMD EPYC 7713 64-Core Processor
model name : AMD EPYC 7713 64-Core Processor
(projects-env)
(tom@localhost)-(jobs:0)-(~/projects/tests)
(! 2007)-> free -m
total used free shared buff/cache available
Mem: 3931 169 2308 0 1453 3493
Swap: 511 15 496
That's 4 gig of ram and 2 CPUs.
Let's try it in C
(! 2010)-> gcc -o test3 test3.c
(! 2012)-> ./test3
First 5 rows:
252 261 234 48 282 Mean: 215.400000
108 261 279 150 66 Mean: 172.800000
189 84 273 180 192 Mean: 183.600000
81 123 81 219 111 Mean: 123.000000
36 207 204 90 249 Mean: 157.200000
Time: 2.101665
2.1 seconds.
It's 2.6 times faster. That's not a lot. Slow would be something that's 100x slower or more than C.
And the C is a hell of a lot more code to do a simple task.
If you've got a vector with 200 million entries, this is still only going to take you 9 minutes to run vs 3.4 minutes with C..
That's nothing.
Even 2 billion is going to be 90 minutes compared with just over half an hour with C.
The C will take much longer to write, and just isn't practical. If it was, then we'd all be using it exclusively.
Python is MORE THAN fast enough.
Most things are going to be network and file IO bound, as I keep saying. This means the bottleneck is the network and file read/write operations like a database.
Micro optimizing the components of most applications will make no difference when it's still going to take the majority of the running time to read or write to a db, or wait for a remote API.
We don't have a speed problem in 2024. If I want to do my operations on a 2 billion row vector then I'll just spin up a 192GB 32 CPU shared linode at $1.728/hr. I'll need significantly more ram for a vector that size anyway, C or no C.
32 CPUs vs 2..
Just for shits and giggles..
root@localhost:~# python3 test3.py
0 1 2 3 4 mean
0 261 153 138 198 45 159.0
1 234 276 18 213 168 181.8
2 81 246 264 144 48 156.6
3 276 183 138 168 48 162.6
4 84 21 45 54 9 42.6
Time: 3.762291669845581
root@localhost:~# free -m
total used free shared buff/cache available
Mem: 193222 389 191476 1 1356 191306
Swap: 511 0 511
root@localhost:~# cat /proc/cpuinfo | grep name | wc -l
32
And the C one on the 32 core
root@localhost:~# ./test3
First 5 rows:
252 261 234 48 282 Mean: 215.400000
108 261 279 150 66 Mean: 172.800000
189 84 273 180 192 Mean: 183.600000
81 123 81 219 111 Mean: 123.000000
36 207 204 90 249 Mean: 157.200000
Time: 1.715726
Of course the C one isn't using more cores. numpy and pandas do, to some extent use more cores/cpus, but obviously not EVERY cpu.
I increased from 2 million rows to 20 million and we get
root@localhost:~# python3 test3.py
0 1 2 3 4 mean
0 216 123 126 162 258 177.0
1 21 51 87 174 204 107.4
2 12 129 3 222 18 76.8
3 135 63 66 105 6 75.0
4 144 252 72 60 180 141.6
Time: 40.48399567604065
Good enough..
So 2 billion would be just over an hour with Python.
That's a LOT OF ROWS, and it would only cost me $1.7 of VPS time.
Now imagine you have massively complex things to write. What are you going to do, write 50x more code in C, taking you 100x longer because it's vastly more complex as well as being more lines, so instead of taking you a week to code, it takes you a year, just so you can run your code in 30 minutes instead of 1.5-2 hours?
So you can save $1 on your VPS by running your C code on a cheaper VPS.
I hope I've shown you here the utter stupidity of a general statement like "python is slow".
Last edited:
- Oct 9, 2013
- 3,471
- 14,452
I don't know what's going on. I couldn't post that C code above. It kept giving me errors. I tried posting it again and it's worked. The forum doesn't seem to like too many funny characters perhaps and it's causing issues.. Anyway, that's the C code under where it says "Let's try it in C"
Last edited:
D
Deleted member 1701249
Guest
absolutely not . python is slow . there is a significant advantage in using low level languages , no company that works in serious projects and not just commercial applications will use python . Python is faster only because it takes the developers less time to create something since packages and optimized procedures already exist and the general structure of the language + the costs are lower . Creating operating systems / serious network infrastructure / industrial systems require precision and optimized algorithms that use less computing resources .
- Oct 9, 2013
- 3,471
- 14,452
What a load of nonsense. For starters, there's no way you read my entire post in 1 minute.
"no company that works in serious projects".
YouTube is written in Python.
So is Netflix.
I made a list above.
*I SHOWED YOU* in code comparing C and Python, when you use the right libraries for the job Python is only a little slower than C.
I then explained to you that in 2024 we don't have a speed problem. We can easily spin up a few more VPS's should we need some extra speed.
"optimized algorithms" ?
And you think a company is going to hire a dev to re-write existing algorithms that are near perfectly implemented and battle tested in production ready libs?
They're going to avoid Python, and instead re-invent the wheel with a low level language and introduce a ton of bugs that they'll never find until it's too late?
That's madness.
Operating systems? What company is creating operating systems? And why do people keep re-hashing this old argument. "Oh, Python can't be used to create an OS, so it's not useful".
We don't need any more OS's. Companies aren't building OS's. Those are done in C and Assembly.
Serious network infrastructure? This doesn't even make sense? What exactly are you talking about? Building network hardware?
Or are you actually telling me that a low level language is better for client-server applications?
Why isn't YouTube written in C? Or Netflix?
Read this - https://www.pythonforengineers.com/why-is-python-so-popular-if-its-so-darn-slow/
I doubt you will, but there you go. Also, actually read what I write instead of just replying by repeating yourself, but in different words.
- Jul 2, 2020
- 806
- 658
Keep it civil. Don't get triggered for criticism.Some people got derailed trying to argue with the poster above me.
This ^ @atmcoder
;-)
- Oct 9, 2013
- 3,471
- 14,452
I get triggered, because I was coding applications before the dotcom bubble. I was having these discussions in the mid 90's with people. I write a really detailed reply with real world, live code examples run on multiple VPS's, in both Python and C, and instead of reading it and discussing those results as they apply to the discussion, he just repeats himself with nonsense about no real companies would use Python.
This is not how you have a discussion.
It's just wild claims after wild claims.
And if any of that "real company" stuff had merit, Python wouldn't even exist. There would have been no jobs, no demand, no nothing. Just low level languages prevailing forever.
This is not how you have a discussion.
It's just wild claims after wild claims.
And if any of that "real company" stuff had merit, Python wouldn't even exist. There would have been no jobs, no demand, no nothing. Just low level languages prevailing forever.
Last edited:
D
Deleted member 1701249
Guest
Lol i didnt get triggered i just had a more serious tone to my message . I have no problem with him its only a conversation , his point is true and my point is true too, python is not as fast as low level languages but its extremely useful and flexible , all i am saying is that in projects that require precision and lower usage of computer resources (such as os development / industrial systems / network operations as i mentioned earlier ) companies will not use python and if they do it will be a bad choice
- Jul 2, 2020
- 806
- 658
Compared to C and C++ there is only one thing I like regarding Python: You can decompile it easily to have a look at the source code (with the help of some deobfuscators of course).
That's not possible with compiled C or C++ code (only disassembling possible).
So people, please continue to use Python, I love to take a look inside. ;-)
That's not possible with compiled C or C++ code (only disassembling possible).
So people, please continue to use Python, I love to take a look inside. ;-)
D
Deleted member 1701249
Guest
yes youtube and netflix are big companies but they belong in the 'commercial' space , apps that people use every day . if you try to create an operating system using python its going to be slow. same with industrial systems . Another example is nasa , you need low level languages . Python is written in a lower level language . why was it ? because low level languages are faster .
D
Deleted member 1701249
Guest
exactly
D
Deleted member 1701249
Guest
This conversation is way to big to analyze here . Google doesnt have an 'official' language they literally program in everything . This is not a good counter argument honestly .
- Jul 2, 2020
- 806
- 658
Another example:
Facebook moved from PHP (a scripting language like Python) via HipHop (PHP to C++ converter) to pure C++ code. There must be a reason why.
- Oct 9, 2013
- 3,471
- 14,452
Also, for anyone that actually read the tests above.
I just noticed that the C code that chatgpt generated wasn't accurate. It cheats.
It doesn't generate the vector, THEN multiply it all by 3.
It multiplies by 3 as it generates.
I fixed the code and reran it and we get
First 5 rows:
252 261 234 48 282 Mean: 215.400000
108 261 279 150 66 Mean: 172.800000
189 84 273 180 192 Mean: 183.600000
81 123 81 219 111 Mean: 123.000000
36 207 204 90 249 Mean: 157.200000
Time: 3.112862
Fixed code is
(tom@localhost)-(jobs:0)-(~/projects/tests)
(! 2016)-> cat test3.c
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
int main() {
int n = 20000000;
int m = 5;
int i, j;
double sum, mean;
clock_t start = clock();
int **array = (int **)malloc(n * sizeof(int *));
for (i = 0; i < n; i++)
array = (int *)malloc(m * sizeof(int));
for (i = 0; i < n; i++) {
for (j = 0; j < m; j++) {
array[j] = (rand() % 100 + 1) ;
}
}
for (i = 0; i < n; i++) {
for (j = 0; j < m; j++) {
array[j] = array[j]*3;
}
}
printf("First 5 rows:\n");
for (i = 0; i < 5; i++) {
sum = 0;
for (j = 0; j < m; j++) {
printf("%d ", array[j]);
sum += array[j];
}
mean = sum / m;
printf("Mean: %f\n", mean);
}
// End time measurement
clock_t end = clock();
double time_spent = (double)(end - start) / CLOCKS_PER_SEC;
printf("Time: %f\n", time_spent);
for (i = 0; i < n; i++)
free(array);
free(array);
return 0;
}
So Python is 80% slower. Not even twice as slow. That's really nothing.
Anyway, I'm done. You guys are not having an intelligent conversation here. It's tiring.
When the only arguments are "nasa uses low level languages", and "I like to look inside your code", I know it's time to depart.
I just noticed that the C code that chatgpt generated wasn't accurate. It cheats.
It doesn't generate the vector, THEN multiply it all by 3.
It multiplies by 3 as it generates.
I fixed the code and reran it and we get
First 5 rows:
252 261 234 48 282 Mean: 215.400000
108 261 279 150 66 Mean: 172.800000
189 84 273 180 192 Mean: 183.600000
81 123 81 219 111 Mean: 123.000000
36 207 204 90 249 Mean: 157.200000
Time: 3.112862
Fixed code is
(tom@localhost)-(jobs:0)-(~/projects/tests)
(! 2016)-> cat test3.c
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
int main() {
int n = 20000000;
int m = 5;
int i, j;
double sum, mean;
clock_t start = clock();
int **array = (int **)malloc(n * sizeof(int *));
for (i = 0; i < n; i++)
array = (int *)malloc(m * sizeof(int));
for (i = 0; i < n; i++) {
for (j = 0; j < m; j++) {
array[j] = (rand() % 100 + 1) ;
}
}
for (i = 0; i < n; i++) {
for (j = 0; j < m; j++) {
array[j] = array[j]*3;
}
}
printf("First 5 rows:\n");
for (i = 0; i < 5; i++) {
sum = 0;
for (j = 0; j < m; j++) {
printf("%d ", array[j]);
sum += array[j];
}
mean = sum / m;
printf("Mean: %f\n", mean);
}
// End time measurement
clock_t end = clock();
double time_spent = (double)(end - start) / CLOCKS_PER_SEC;
printf("Time: %f\n", time_spent);
for (i = 0; i < n; i++)
free(array);
free(array);
return 0;
}
So Python is 80% slower. Not even twice as slow. That's really nothing.
Anyway, I'm done. You guys are not having an intelligent conversation here. It's tiring.
When the only arguments are "nasa uses low level languages", and "I like to look inside your code", I know it's time to depart.
D
Deleted member 1701249
Guest
by real company/organization i mean sectors with more important usages of programming , industrial , military systems , healthcare , os developing , scientific research . Aka things that require extreme precision and lower usage of computer resources . Python is used by more people and there is such a high demand because its a lot easier to grasp .
D
Deleted member 1701249
Guest
80% of speed in the tech industry is translated into billions of dollars . I think you are the one getting really triggered and making an 'unintelligent' conversation .
D
Deleted member 1701249
Guest
i do NOT like php , i can see why they changed lol
- Oct 9, 2013
- 3,471
- 14,452
And just because I like to point out your constant factual errors.
https://github.com/facebook/hhvm/wikihttps://en.wikipedia.org/wiki/HHVMhttps://en.wikipedia.org/wiki/HipHop_for_PHP
Facebook didn't in fact convert all their PHP code to C++.
You're just making stuff up to support your argument.
Their original codebase probably sucked and they needed to improve performance back in 2010-2011.
They dabbled with a few ideas, one of them converting parts of their codebase to C++ for performance.
But my god, if you understood software engineering you'd realize you can't just convert an entire codebase like that and be done. It's not practical. The resultant C++ code is not human-readable/maintainable.
That's why they shifted to developing HHVM
"
By many accounts HPHPc fulfilled its goals, especially within Facebook as it allowed facebook.com to run much faster while using fewer resources. However, in early 2013 Facebook deprecated HPHPc in favor of the HipHop Virtual Machine (HHVM), which is a just-in-time (JIT) compilation-based execution engine for PHP, also developed by Facebook.[2][9] There were many reasons for this; one of them was HPHPc's flattened curve for further performance improvements. Also, HPHPc did not fully support the PHP language, including the create_function() and eval() constructs, and it involved a specific time- and resource-consuming deployment process that required a bigger than 1 GB binary to be compiled and distributed to many servers in short order. In addition, maintaining HPHPc and HPHPi in parallel (as they needed to be, for the consistency of production and development environments) was becoming cumbersome. Finally, HPHPc was not a drop-in replacement for Zend, requiring external customers to change their whole development and deployment processes to use HPHPc.[2]
"
Please, just stop making stuff up. You'll never progress as a developer with that attitude.
No, it's not billions of dollars. VPS's are cheap.
The python on a $24/mo VPS beats the C on a $7/mo vps.
Is that billions of dollars?
https://www.blackhatworld.com/seo/10-years-coder.1561348/#post-17200272
You're impressed by someone who's been coding for 10 years. I've been in the game for 30.
Seriously enough. I'm out.