Skip to content



  1. Introduction:
  • Nowadays, Piwik is the best open-source web analytics. It could be easy to use, simple to install and has a lot of customization. Many companies used it within their cooperation. With Piwik, people could easily to have in-house analytics tool with all their data.
  • Piwik could easily to handle 100.000 pageviews a day. The Piwik’s bottleneck is database resource for archive processing. I made Piwik to handle 40 millions requests a day. But with special customization – Oracle Cluster Database, Tracking Queue, Database sharding. These customization need a lot of development and system administration work. There is no effectiveness way to make Piwik has scalability.
  • BigQuery is a low cost enterprise data warehouse for analytics. That can easily to handle GB to PB data, without worry about scaling. BigQuery standard SQL is compliant with the SQL 2011.
  • As BigQuery cost is based on scan size of data. We could reduce the cost of running database cluster. 1TB for just $5, that is the amount need to processing 50GB a day (24 time running archive processing). With my own experience we need 4 server with 16 CPU core and 32GB Ram to handle such large data each day which might cost $2000 a month with AWS (and more if you use your own servers).
  • This project purpose is make a Piwik installation that could run easily on Google cloud environment with BigQuery as core Archive Processing. We might have our own analytics software running on Google Infrastructure, the same as current Google Analytics 360 offer without payment $150.000 a year to get our own raw data.


System Architecture


Chrome (browser) Extension might hack your bank

Chrome and Extension

  • Chrome is the gateway to interact with the Internet worlds.
  • People trust their browser, Extended Validation SSL, bank’s website, Windows, Mac, ISP… but the third party extension could to many dangerous without any notice to user.
  • Hacker could control everything behave of user just using an extension.

Ultimate permission of Extension.

  • Look at this permission, is it dangerous?Screen Shot 2017-06-24 at 10.19.40 PM
  • Extension could see what user’s seeing, know what user entering… And create a completely fake webpage with this permission.  It’s called Javascript Injection attack.


  • An online banking service with SMS OTP verification for online money transfer.
  • Simple POC using javascript and inject via an Chrome’s extension.

Screen Shot 2017-06-24 at 10.12.41 PM.png

  • Fake fields are inserted into the HTML, real input with hacker’s account number fields are hidden.
  • SMS OTP is by-passed because of no destination account in the SMS content.

  • The OTP was used for another transaction.



  • If you don’t know anything. Don’t install any plugin/extension or software that install a extension.
  • If you are advance user, using web browser incognito mode or anther browser with no plugin enabled for bank transaction. We can trust no-one in the Chrome Extension Web Store.


  • Add destination account to the OTP.
  • OTP token hardware with transaction signing


Google Chrome:

  • More secure permission system.
  • Ask permission to edit HTML content for each website.






Get the log line with date range

If you want to extract log with prefix is date time like this

[2017-05-01 02:00:06,300] this is a log line details

We might use the awk command following with the condition

awk -F'[]]|[[]'   '$0 ~ /^\[/ && $2 >= "2017-05-01 00:00" { p=1 } $0 ~ /^\[/ && $2 >= "2017-06-01 00:00" { p=0 } p { print $0 }' details.log >  2017-05.log



Google Maps traffic, It is not “The world’s local navigation”

There is no debate about the market share of the map and navigation anymore. Google Maps beat all other competitor by the Android Phone. Apple navigation is some kind of useless in Vietnam, it shows 32minutes for 28km through the centre, Google map tells 1hh15, more feasible. Other competitors have own advantages, Vietmap, Vietbando, Here, sygic, but no one has released the same function based on user collected data, realtime traffic data.


3897227_google-maps-1Traffic data – image:

While the traffic data is a useful reference information, we can not rely on this for navigation. Why?  But it still not something that can be used while driving, it’s not safe to check on the screen rather than focus on the road, where any moto-bike might appear from any direction on anytime.

The voice control is just for some one doesn’t know clearly about the city and need turn to turn direction. It doesn’t give any recommendation rather estimation when we arrive.

The traffic in Vietnam is chaos, any algorithm which is success on other places can’t be applied in here. Even Hanoi and Ho Chi Minh City have different driving regulations.

From my point of view, with experience of more than 10 years driving through the city, I have some idea to make a useful local navigation which could help traffic in Vietnam a lot.

What do I need?

I drive a lot. My Moto bike reached 50.000km for 4 years. 35km everyday, just go around the city 🙂 I live in the north of city, but sometime I worked in the south). I knew many route, knew how the city’s traffic growing, and some how, how people driving in the city. I have a good habit, be on time. Based on my experience, I might predict exactly which route I should go to the destination to be on time. But it’s just for moto-bike, I’m driving a car now, there are many factors could affect the journey…

  1. I need a navigator help me to be on time.
    It’s not just how I can drive to the destination. The importance is when I’ll arrive. The Google Now has remind to leave, but I can’t rely on this one. I has experience that Google map told me it’s take 40 minutes to the destination, actually more than 1 hours.
  2.  I need an assistance. 
    Rather than just giving me option to choose, I want an assistance should choose the route for me. I have total 4 main routes which could lead me to the office. But the shortest is not the fastest, it based on when I start, which weekday, any special construction, or just a back to school day :).
  3. A precious time to start 🙂
    Believe me or not, sooner is not better (see the shortest is not fastest). There is some time frame in the date that some route/place is peaceful. If we choose exactly time to start with careful prediction, we might save time and money.  Go to work at 5am that I could be at the office at 5am55 but it is not a good idea.
  4. Crowdsourcing traffic data
    It’s not just live traffic data, It seem useless on the road for car, we can’t not easily to change the route while we on a traffic jam. We should predict it and prevent it happen.
    People have routine, it might change overtime, but I’m sure that most of the people would choose the same route for same destination for daily schedule. Go to work, go home, even the taxi, uber, which based on the passenger pattern.  Read through the people mind, we might predict exactly what would happen.
    With user data, we might guess exactly the vehicle, (moto-bike which is most usage but not available on Google maps, car, bus, taxi).
    Based on the traffic data, we might upgrade the Map data automatically, forbidden road, one way road,  car turn forbidden based on time, speed limit, internal roads …
  5. Know me better
    It should know how I drive, do I prefer this road.
  6. All in one place to help a better live
    Imagine that Google Assistance, ask you to go to bed sooner if you want to wake up at 6am, start driving no later than 6:45 to be at the meeting at 8. You might drive 15 sooner for most pleasure driving and save the gasoline or choose the highway for fastest road.
    30 minute later. Hey you, it seem like we have an accident in the next corner, don’t worry, I will help you out, make an u-turn an turn right at the crossroad  (just like realtime satellite navigation on action movies) . It seem like you would be late for the meeting. Do you want me to make a call to re-arrange?

What can I do?

Google can not know the city better than me. Although I can’t not access to the raw traffic data or the Google user history. But I could provide a myself experience based on location history. Use my knowledge of algorithm to help myself to to work. If I’m lucky enough, I could ask my friend from FPT to provide some traffic data based on CCTV, which some kind of a city project.

Let wait what I can do.

ulimit trên MacOS, như ác mộng

Làm đủ mọi cách vẫn bị cái ulimit nó hành. Cuối cùng kiếm ra được cái Link về vấn đề này trên MacOS, Apple nó chơi xấu developer.