Wednesday, November 16, 2016

django-cron running command constantly instead of on schedule

Hi python-lovers,

Today I've experienced weird problem with django-cron==0.4.6 (and django 1.9.5). And I thought it would be useful to share with you.

I had job class which is defined like this:
class MyJob(CronJobBase):
   RUN_EVERY_MINS = 60 * 24    
   schedule = Schedule(run_every_mins=RUN_EVERY_MINS)
   code = 'myapp.myjob'    
   def do(self):
       do_some_task()

Theoretically it should be run once a day. But in practice, every time when I was trying to run "python manage.py runcrons", do_some_task() had been called.

I tried everything, including downgrading django-cron. But the actual problem was that do_some_task had an exception in it. Unfortunately it was not reported in any way. No logs, no warnings were spit by django-cron.

The conclusion

If there is any exception in your job, it will be re-run unlimited number of times by django-cron, ignoring your schedule. Beware of that.

Sunday, November 13, 2016

How to install java 8 in Debian Jessie from Oracle website

Hi all,

Sometimes it's necessary to install newer version of java that there is in official Debian Jessie repository. In that case you need to download it from oracle website, extract files and update links to java executables. It can be done via update-alternatives or simply by replacing links (which is not recommended). Below I will show you both methods.


1) Update-alternatives way


1) create install folder and navigate to it:
mkdir ~/install
cd ~/install

2) download jdk 1.8. The tricky part here is that you need to accept license agreement by passing a cookie:
wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u73-b02/jdk-8u73-linux-x64.tar.gz

3) extract files from archive:
tar -xvzf jdk-8u73-linux-x64.tar.gz

4) Install alternative for each java executable that you need to use. In this example we will install 3 alternatives for: java, javac, jar (but you may also install more than just 3):
sudo update-alternatives --install /usr/bin/java java ~/install/jdk1.8.0_73/bin/java 1000
sudo update-alternatives --install /usr/bin/javac javac ~/install/jdk1.8.0_73/bin/javac 1000
sudo update-alternatives --install /usr/bin/jar jar ~/install/jdk1.8.0_73/bin/jar 1000
The last number in the command is the priority which is used in auto-mode only, where program with the highest priority is used as default. If you want to understand the difference between manual and auto modes, I recommend watching this video.

5) Configure alternatives:
sudo update-alternatives --config java
sudo update-alternatives --config javac
sudo update-alternatives --config jar
After running each command, change the selection by entering a number, which is manual mode of particular executable in jdk1.8.


2) Manual way (not recommended)


2.1) jdk 1.8


1) Download jdk and extract files as described in 1.1-1.3.

2) update links to point to java 1.8 (you may need to backup original files):
sudo ln -sf ~/install/jdk1.8.0_73/bin/jar /usr/bin/jar
sudo ln -sf ~/install/jdk1.8.0_73/bin/javac /usr/bin/javac
sudo ln -sf ~/install/jdk1.8.0_73/bin/java /usr/bin/java

#sometimes you may need to update default-java
sudo mkdir -p /usr/lib/jvm
sudo rm /usr/lib/jvm/default-java
sudo ln -s ~/install/jdk1.8.0_73/bin/java /usr/lib/jvm/default-java

2.2) jre 1.8


If you don't need java compilers, then you can only download jre.

1) Download jre and extract files as described in 1.1-1.3.

2) update links to point to java 1.8 (you may need to backup original files):
sudo ln -sf ~/install/jre1.8.0_73/bin/java /usr/bin/java

#sometimes you may need to update default-java
sudo mkdir -p /usr/lib/jvm
sudo rm /usr/lib/jvm/default-java
sudo ln -s ~/install/jre1.8.0_73/bin/java /usr/lib/jvm/default-java


The end


That's it. After completing these steps using either 1st or 2nd method you should have working java 8 on your Debian Jessie machine. To check current java version use this command:
java -version
which should return something like this:
java version "1.8.0_73"
Java(TM) SE Runtime Environment (build 1.8.0_73-b02)
Java HotSpot(TM) 64-Bit Server VM (build 25.73-b02, mixed mode)

You can also watch this tutorial:

Tuesday, November 1, 2016

Launch of RuDividends - Russian Dividend Stocks website for Investors

Hi everyone,

Today I won't tell you much about Python, but I will tell you about what I've done a while ago using Python. I've been hard at work (mostly at nights and in my spare time) developing my new service, called RuDividends which helps investors across the world analyze dividend stocks in Russia.

The website lists Russian stocks sorted by highest dividend yield. Also you can find all of the dividend history of any stock in one place both in Russian and English languages.

Why did I decide to create this service? First of all, I'm from Russia. Then when I tried to find any good website with all dividend stocks, I could barely find it in Russian and nothing in English. And more important, I created it for my own use, because I invest some funds in Russian companies and I need to analyze dividends anyway.

I also believe that Russia has huge amount of natural resources and most of its companies are undervalued. Some of the companies are paying very high dividends (risk is also high though). Moreover, this year (2016) Medvedev signed an order mandating 8 state-controlled companies to pay at least 50% of their profits in dividends, which helps to plug a hole in the state budget.

All of this shows that you can benefit by investing in Russian companies. Hope you'll find this service useful. And I'd really appreciate if you could spread the word to as many other investors as possible.

Happy dividend hunting!

Friday, June 17, 2016

pg_dump version mismatch - How to fix in CentOS 7

If you've encountered with this issue:

pg_dump: server version: 9.4.7; pg_dump version: 9.2.15
pg_dump: aborting because of server version mismatch

Then it means that the version on your local pg_dump tool is older that the version of the remote postgresql database.

In my case I had postgresql 9.2 running on CentOS 7 and I was trying to dump database out of remote postgresql 9.4. So the solution was to install 9.4 using postgresql repository instead of centos distro repository. Below you will find instructions that you need to do in order to fix this issue on CentOS.

1. Navigate to this link and choose necessary rpm:
http://yum.postgresql.org/repopackages.php#pg94

2. Install rpm
sudo rpm -Uvh https://download.postgresql.org/pub/repos/yum/9.4/redhat/rhel-7-x86_64/pgdg-centos94-9.4-2.noarch.rpm

3. Install 9.4:
sudo yum install postgresql94

4. Find necessary pg_dump location:
find / -name pg_dump -type f 2>/dev/null

5. Run newly installed pg_dump:
/usr/pgsql-9.4/bin/pg_dump -h {dbhost} -U {dbuser} {dbname} > {dbname}.sql

That's it :-D

Saturday, February 20, 2016

Django prefetch_related VS select_related in a single database query

Imagine the following model:
class Parent(Model):
    ...

class Child(Model)
    parent = ForeignKey(Parent)
    ...

I want to list all parents along with first related child.

Thursday, February 4, 2016

Cracking wifi password with pyrit and NVIDIA GPU on Amazon AWS

WPA algorithm is very secure, and to get the password usually we have only one way - to brute force it, which could take huge time if password is strong enough. But what if instead of using regular CPUs we would use a power of GPU? Amazon says, that we can use up to 1,536 CUDA cores on g2.2xlarge instance, which costs $0.65 per Hour. Sounds very promising, so let's see how it can help us to speed up password brute force.

Below I will give step-by-step tutorial on how to deploy Amazon GPU instance and run pyrit (python tool) to crack password using GPU.

Tuesday, February 2, 2016

Tricky questions: function parameters with default mutable values

Today I discovered one confusing feature of Python. And also decided to open new category on my blog, which will be called "tricky python questions".

def append(num, to=[]):
    to.append(num)
    return to

aa = append(1)
print aa

bb = append(2)
print bb