Connect to Other Google Cloud Services in Google Compute Engine (Go Language)

    It has been discussing for a long time: Should I use Compute Engine or App Engine? Well, If you need full control on the system, it's definitely Compute Engine. But If you just want to build an application system, including dynamic web server, API server and database, for example, it would be pretty hard to choose between them. Although this article is not going to do some real comparison, the following may give you some concepts on the differences between the two platforms when it comes to building application systems.

    Connecting to other google cloud services within the App Engine is a piece of cake:

func handle(w http.ResponseWriter, r *http.Request) {
    c := appengine.NewContext(r)

    e1 := Employee{
        Name:     "Bekket McClane",
        Role:     "Engineer",
        HireDate: time.Now(),
    }

    key, err := datastore.Put(c, datastore.NewKey(c, "employee", "mshockwave", 0, nil), &e1)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    var e2 Employee
    if err = datastore.Get(c, key, &e2); err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    fmt.Fprintf(w, "Stored and retrieved the Employee named %q", e2.Name)
}

    All we need to do is instantiating a new context and call the straightforward Put and Get functions. Most of the google cloud service usages are as easy as this case, except the cloud storage, which we would mention the reason later.
    Let's look at the Compute Engine side. The Compute Engine, unlike App Engine, which is wrapped in a well managed runtime, accesses other google cloud service as if a normal client-side application. In another word, your machine need to handle all the OAuth2 flow with the google api client library. What's worse, there is no stable release of Go language SDK specific for cloud platform. But it's worth for a shot to use the github version of Go cloud platform SDK. Here is a simple datastore example(You can still use go get to fetch related libraries):

package storage

import (
 "golang.org/x/oauth2/google"
 "golang.org/x/net/context"
 "log"
 "google.golang.org/cloud/datastore"
 "google.golang.org/cloud"
)


var datastoreClient *datastore.Client
var ctx context.Context
func init(){

 ctx = context.Background()

 defaultTokenSource, err := google.DefaultTokenSource(ctx,
  datastore.ScopeDatastore,
  datastore.ScopeUserEmail,
 )
 if err != nil || defaultTokenSource == nil{
  log.Fatalf("Error getting google default token source")
  return
 }

 datastoreClient, err = datastore.NewClient(ctx, PROJECT_ID, cloud.WithTokenSource(defaultTokenSource))
 if err != nil {
  log.Fatalf("Error getting datastore client")
  return
 }
}

func GetDatastoreClient() *datastore.Client { return datastoreClient }
func GetDatastoreContext() context.Context { return ctx }

    Authorization part is always out first concern, the approach I used in the above snippet is the neatest in my opinion. Since the authorization way in the official go document use the json key file(you can retrieve one in your developer console) which is less portable. The key component works behind DefaultTokenSource is the gcloud utility in  Google Cloud SDK. It would set some environment variables holding authorization informations. Google do really well on this part since it not only keep you from handling the messy authorization flow, it also let you use the same code both in the cloud platform AND local development environment. So you really need to install the Google Cloud SDK no matter what kind of application you are building. There are other authorization methods which is in the google package of the official go oauth2 library.
    It seems that Google is going to move cloud platform related parts from its API client library to individual cloud platform library which is still under development so far. The similar situation happens on App Engine since they are trying to unify the libraries between App Engine and the new Managed VM. Some of the important parts, Blob service for example, are going to be deprecated. They want to remove most of the Blob service functions and migrate all file related operations to the cloud storage. Thus you can find that the usage of cloud storage service is different from other google cloud services due to the progressing migration and library integration with Managed VM.

Android State Machine Library

    In a "code surfing" within the AOSP WIFI subsystem, I found that there is a great and well developing state machine library working behind it. Although it is hidden by the build system so we can't use it from SDK, there are ways like my previous article to access them. This article shows a brief how-to due to lack of documentation of the library.

Introduction

    State machine plays an extremely important role in computer science. In short, the machine would stay in a specific state at any moment and transfer to another state if a particular condition is satisfied. There are also lots of variant like the most familiar, to college student who just finished the discrete math class, Finite State Machine, and another one which is used in this case, Hierarchy State Machine. As the name implies, there is a hierarchy structure among the states within the machine like the below diagram.

(Fig. 1)

State B1, B2 and B3 are the children of state B. These state still have the properties of normal state like transition among each other if certain condition happens, but the hierarchy relationship would make the execution logic a lot different which we'll see later.

Usage

    If you feel lazy to fetch the internal Java API of AOSP following the previous suggestion, here is a jar containing some internal classes which are mostly in com.android.* or com.google.* packages, state machine classes we are discussing are also in that. You had been warned that the jar was build from AOSP master branch which API level was about 21 then(Although I think the APIs within it are mostly platform-neutral).

    Classes we're going to use are StateMachine and State which live in the com.android.internal.util package within platform/framework/base repository. The truth is that there IS a simple documentation of the machine design embedded as code comment within the StateMachine source code. I had converted it to web page and you may read it from here. 

    First, we need to create states by extending State class.

class A extends State {
 @Override
 public void enter(){
  Log.d(TAG, "Enter state A");
 }

 @Override
 public boolean handleMessage(Message msg){
  boolean ret = NOT_HANDLED;
  switch(msg.what){
   case CMD1:
    sendMessage(mSM.obtainMessage(CMD2));
    ret = HANDLED;
    break;

   case CMD2:
    transitionTo(mStateB);
    ret = HANDLED;
    break;
  }

  return ret;
 }

 @Override
 public void exit(){
  Log.d(TAG, "Exit state A");
 }
}

Although the handleMessage method is not abstract, it must be implemented. Other like enter or exit are optional. The state machine is a Moore machine, that is, actions would be performed after migrating to the destination state instead of the transition process. The most powerful thing, in my opinion, is the message delivering mechanism which handleMessage is responsible for. In the above example, state A send CMD2 upon receiving the CMD1 message, and since we're still at state A, the CMD2 case would be triggered and migrate to state B. But what if we get messages neither CMD1 nor CMD2, that is, we're NOT_HANDLED ? Let's look at the below example and take Fig. 1 as design blueprint.

class B extends State {
 @Override
 public void enter(){
  Log.d(TAG, "Enter state B");
 }

 @Override
 public boolean handleMessage(Message msg){
  boolean ret = NOT_HANDLED;
  switch(msg.what){
   case CMD3:
    Log.d(TAG, "Get CMD3");
    ret = HANDLED;
    break;
  }

  return ret;
 }
}

class B1 extends State {
 @Override
 public void enter(){
  Log.d(TAG, "Enter state B1");
 }

 @Override
 public boolean handleMessage(Message msg){
  boolean ret = NOT_HANDLED;
  switch(msg.what){
   case CMD1:
    Log.d(TAG, "Get CMD1");
    ret = HANDLED;
    break;

   case CMD2:
    deferMessage(msg);
    transitionTo(mStateB2);
    ret = HANDLED;
    break;
  }

  return ret;
 }
}

If we send CMD3 when the machine is at state B1, since state B1 couldn't handle it, the machine would leave state B1 and deliver the message to B1's parent, state B. So the output after sending CMD3 would be "Enter state B" followed by "Get CMD3". Clever person like you must find out another interesting thing: deferMessage. The deferMessage would store the passed in message and send it to the next state after migrating. If there are multiple accumulate message, oldest one would deliver first and then so on.
    Finally, we need to add all the states to the machine and define the relations, still take Fig. 1 as structure blueprint.

mSM.addState(mStateA);
mSm.addState(mStateB);
 mSM.addState(mStateB1, mStateB);
 mSM.addState(mStateB2, mStateB);
 mSM.addState(mStateB3, mStateB);

mSM.setInitialState(mStateA);

    Now we're going to talk about the hierarchy design's affection on machine behavior. If the machine start from state B1, the execution flow actually won't execute from B1, instead, it would first pass through state B, and finally state B1. As a consequence, if the initial state is B3, the execution flow would be: B -> B1 -> B3. Same philosophy can be applied when we migrate from one "tree" to another "tree". That is, if we called transitionTo(mStateB2) when we're at state A, the flow after transition would still be: B -> B1 -> B2.

Summary

    I'm not an expert in discrete math, so I don't know the real theoretical advantage of hierarchy state machine. But in my opinion, as an developer, this sort of design pattern is suit for situation where we must do A before B. In another word, HSM makes guarantee on the processing order among some of the tasks. Thus, Android system use it on places like WIFI subsystem and some of the Bluetooth profile subsystems like A2DP.

[Quick Note] Android JobScheduler

Android API 21 新加入了 JobScheduler API 以及他的朋友們，目的就如字面上的意思，可以允許開發者排定一個定期工作，取代原有的 Alarm Manager + Broadcast Receiver 方式。

我第一眼看到 API 是覺得還蠻新奇的啦
setPersisted()：開機後可以繼續活著

setRequiredNetworkType(), setRequiresCharging(), setRequiresDeviceIdle()：喔喔！這個不錯，終於可以設定條件了嗎？！

雖說可以依照電量或機子狀態做選擇，但效能部分呢？會不會只是 Alarm Manager 的再包裝版呢？

先來看看常見的 Alarm Manager + Broadcast Receiver 方法：

1. App 必須要開一個 Service，持續運行在背景
2. Service 要註冊一個 Alarm Broadcast Receiver，接收 Alarm Service 的broadcast
3. 向 Alarm Service 請求定期發送 broadcast

缺點一當然就是彈性不夠，不像 JobScheduler 可以設定條件，但我認為最致命的就是你必須開一個 Service 持續在背景運行。以使用者的角度，有一個東西在背景佔記憶體當然不怎麼高興，以開發者的角度，最糟的情況是如果機子記憶體很小，你的 Service 很容易因為 Low Memory 被殺掉了。

那 JobScheduler 怎麼做呢？Reference 這樣講過：

When the criteria declared are met, the system will execute this job on your application's JobService.

基本上我們邏輯的部份還是要繼承 JobService 呀，至於 JobScheduler 的 schedule(JobInfo) 吃的 JobInfo 事實上只是個加上資訊的包裝罷了，因為我們在調用 JobInfo.Builder(int, ComponentName) 時，還是要傳入 ComponentName 這個需要 JobService 建構的參數呀。
稍微翻了一下原始碼，終於在 android/server/job/JobServiceContext.java(看來他的邏輯還是簡單地劃出一個執行的 context) 找到了實際執行的函式。

142    /**
143     * Give a job to this context for execution. Callers must first check {@link #isAvailable()}
144     * to make sure this is a valid context.
145     * @param job The status of the job that we are going to run.
146     * @return True if the job is valid and is running. False if the job cannot be executed.
147     */
148    boolean executeRunnableJob(JobStatus job) {
149        synchronized (mLock) {
150            if (!mAvailable) {
151                Slog.e(TAG, "Starting new runnable but context is unavailable > Error.");
152                return false;
153            }
154
155            mRunningJob = job;
156            mParams = new JobParameters(this, job.getJobId(), job.getExtras(),
157                    !job.isConstraintsSatisfied());
158            mExecutionStartTimeElapsed = SystemClock.elapsedRealtime();
159
160            mVerb = VERB_BINDING;
161            scheduleOpTimeOut();
162            final Intent intent = new Intent().setComponent(job.getServiceComponent());
163            boolean binding = mContext.bindServiceAsUser(intent, this,
164                    Context.BIND_AUTO_CREATE | Context.BIND_NOT_FOREGROUND,
165                    new UserHandle(job.getUserId()));
166            if (!binding) {
167                if (DEBUG) {
168                    Slog.d(TAG, job.getServiceComponent().getShortClassName() + " unavailable.");
169                }
170                mRunningJob = null;
171                mParams = null;
172                mExecutionStartTimeElapsed = 0L;
173                removeOpTimeOut();
174                return false;
175            }
176            try {
177                mBatteryStats.noteJobStart(job.getName(), job.getUid());
178            } catch (RemoteException e) {
179                // Whatever.
180            }
181            mAvailable = false;
182            return true;
183        }
184    }

第162~165行就是大部份的答案了：JobScheduler 還是會生出一個 Service 的，但這個 Service 很特殊，也就是 JobService，有 onStartJob(), onStopJob()，不會一直在後台運行的，我個人感覺，JobService 比較像定義一個 task，在對的時間點執行，執行完 task 就結束，讓它比傳統需要一直開 Service 然後等 alarm 的方法更加省資源，也比較直觀。
    那這樣每一個工作就要 spawn 一個 process 的做法，代價是不是有點高呀？畢竟創造進程不是一件小事。
    我也有想過說要不要讓開發者可以把輕量級的工作丟到一個進程，在一個進程裡統一執行。
    恩...我是認為由於 Android App 的模型裡，App 進程已經跟 zygote 共用很多 address space 資源了，因為不管是 activity 還是 service 都是從 zygote fork 過來的，所以會比較像創一個 thread 而不是 process，代價比較輕一些。至於統一執行的部分，實作上會用像是 v8 isolate 的概念，在同一個進程裡區分不同工作的 context，但我覺得不會這樣做是因為 android 的 sandBox system 是用獨立出 process + uid / pid 等方式來做隔離，嘛...遵照一下現有的安全模型還是比較統一啊。

What is hardware cache line ?

When I was learning the SLAB allocator in the Linux kernel, there was always a question in my mind:

WAHT IS "CACHE LINE" ?!

I know CPU has caches, like L1, L2, etc. But why there is a "line" in the name? Moreover, when I was searching for the answer, I encountered more unfamiliar names like "2 way sets cache" or "Fully Associated cache" , and I was confused by those proper nouns, too.

In short, the reason it is called "line" is that a cache needs to be organized in groups, or blocks, and each of them is called "a cache line". What's more important is the way how it's organized. The first method comes out in our mind is to simply divide a memory address(32 bits, for example) into two fields: field at the most-significant-bits side indicates the cache line index, and the other field, the least-significant-bits side, represents the offset within a cache line. Although this sounds pretty straight forward and there do exist a kind of cache organization which is similar to the above description, it still needs to be slightly fixed before putting into practice.

The first kind we are going to introduce, is Fully Associated  cache.

This approach simply split a 32 bits address into 28 + 4 bits. The prior 28 bits field represents the memory block id, so there will be 2^28 memory blocks in this case. As mentioned above, this sort of organization is straight forward, however, it's difficult to implement since every time we want to find a specific memory block among the cache lines, it takes too many comparisons and efforts due to its large tag field, a memory block may locate in any of the cache line.

Before we introduce the solution how most of the modern processors fix the above problem, let's look at the other extreme compared to fully associated cache: Directed Mapped cache

There are some assumptions which are showed at the top-left corner in the picture above. This organization approach is just like its name - it directly maps each memory block to the cache. So the 19 bits field is the id of a memory block and the 9 bits field acts like a "cache line offset within a memory block" which also represents the index of a cache line in the cache. What is the downside of this approach? Well, let's take a look at a vivid example I saw in a lecture web page in Colorado University's ECEE department: If we are going to add two arrays and store the result to the third one which their sizes are all happen to be 16 bytes x 512, since each array perfectly fits the entire cache and each element in the array also perfectly fits a cache line, we might get cache miss on EVERY array access during the calculation!!

Let's look at the usual approach most of the modern processors adopt, thus, the N-Way Sets cache. Here is the 2 way sets:

The N-Ways Sets approach is really similar to the directed mapped approach, that's why I introduced the latter first. If you split the cache into half in the directed mapped cache and "merge" two parts into one by putting them parallel, then you almost get a 2 way sets cache! That is, a cache line now can contain two "slices"(16 bytes, in this case) of memory block which have the same cache line index, and the tag field is charged to identify which memory block this "slice" belongs to.

Using Binder To Vibrate - Android Binder 實戰 (Client Side, Second Part)

    在第一部分，我們幾乎沒有提到 AIDL 的事情，也沒有解釋為什麼 IMyVibrator 要寫成那樣，在這部分就要為大家來揭曉。

    在講 AIDL 之前，先來提一下 Android 的 Java Service 吧，或者更明確的說，bindService() 這個函式與相關的東西。我剛開始學寫 Service 時，常常困惑：假如我要跟 Service 溝通，不就弄一個 public 的變數或函式，讓雙方可以互相存取，為什麼還莫名其妙地跑出一個  onBind() 以及 bindService() 呢？的確，如果是在同一個 package 裏，是可以這麼做的，原因很簡單：在這個情況下，Service 跟主 App 是在同一個 thread。到這邊應該可以大概想出來，常與 bind 相關一起提到的 AIDL 是用在什麼情況了吧？沒錯，就是跨進程的 Service，所以我才在上一篇說到，AIDL 是 Binder 的 Java 綁定版本相關事物。

AIDL 真面目

    事實上 AIDL 檔在 App 的 Build 過程中，會編譯成一個 Java 檔，例如這次例子中的 IVibratorService.aidl ( 位於 frameworks/base/core/java/android/os/ )：

package android.os;

/** {@hide} */
interface IVibratorService
{
    boolean hasVibrator();
    void vibrate(int uid, String opPkg, long milliseconds, int usageHint, IBinder token);
    void vibratePattern(int uid, String opPkg, in long[] pattern, int repeat, int usageHint, IBinder token);
    void cancelVibrate(IBinder token);
}

我們把內容複製下來，改一下 package，貼到一個一樣名字的 AIDL 檔，並且執行有關 AIDL 的 gradle 任務，如下圖所示：

接下來會在 app/build/generated/source/aidl 底下看到生成的 Java 檔。

(綠色的那個 IVibratorService)

把它打開來看，以下是已經排版過的。(盡量不要修改該檔案)

package com.test.vibratorbinder;
// Declare any non-default types here with import statements

public interface IVibratorService extends android.os.IInterface {
 
 /** Local-side IPC implementation stub class. */
 public static abstract class Stub extends android.os.Binder implements com.test.vibratorbinder.IVibratorService {
  private static final java.lang.String DESCRIPTOR = "com.test.vibratorbinder.IVibratorService";
  
  /** Construct the stub at attach it to the interface. */
  public Stub() {
   this.attachInterface(this, DESCRIPTOR);
  }
  
  /**
   * Cast an IBinder object into an com.test.vibratorbinder.IVibratorService interface,
   * generating a proxy if needed.
   */
  public static com.test.vibratorbinder.IVibratorService asInterface(android.os.IBinder obj) {
   if ((obj == null)) {
    return null;
   }
   android.os.IInterface iin = obj.queryLocalInterface(DESCRIPTOR);
   if (((iin != null) && (iin instanceof com.test.vibratorbinder.IVibratorService))) {
    return ((com.test.vibratorbinder.IVibratorService) iin);
   }
   return new com.test.vibratorbinder.IVibratorService.Stub.Proxy(obj);
  }

  @Override
  public android.os.IBinder asBinder() {
   return this;
  }

  @Override 
  public boolean onTransact(int code, android.os.Parcel data, android.os.Parcel reply, int flags)
  throws android.os.RemoteException {
   switch (code) {
    case INTERFACE_TRANSACTION:
     {
      reply.writeString(DESCRIPTOR);
      return true;
     }
    case TRANSACTION_hasVibrator:
     {
      data.enforceInterface(DESCRIPTOR);
      boolean _result = this.hasVibrator();
      reply.writeNoException();
      reply.writeInt(((_result) ? (1) : (0)));
      return true;
     }
    case TRANSACTION_vibrate:
     {
      data.enforceInterface(DESCRIPTOR);
      int _arg0;
      _arg0 = data.readInt();
      java.lang.String _arg1;
      _arg1 = data.readString();
      long _arg2;
      _arg2 = data.readLong();
      int _arg3;
      _arg3 = data.readInt();
      android.os.IBinder _arg4;
      _arg4 = data.readStrongBinder();
      this.vibrate(_arg0, _arg1, _arg2, _arg3, _arg4);
      reply.writeNoException();
      return true;
     }
    case TRANSACTION_vibratePattern:
     {
      data.enforceInterface(DESCRIPTOR);
      int _arg0;
      _arg0 = data.readInt();
      java.lang.String _arg1;
      _arg1 = data.readString();
      long[] _arg2;
      _arg2 = data.createLongArray();
      int _arg3;
      _arg3 = data.readInt();
      int _arg4;
      _arg4 = data.readInt();
      android.os.IBinder _arg5;
      _arg5 = data.readStrongBinder();
      this.vibratePattern(_arg0, _arg1, _arg2, _arg3, _arg4, _arg5);
      reply.writeNoException();
      return true;
     }
    case TRANSACTION_cancelVibrate:
     {
      data.enforceInterface(DESCRIPTOR);
      android.os.IBinder _arg0;
      _arg0 = data.readStrongBinder();
      this.cancelVibrate(_arg0);
      reply.writeNoException();
      return true;
     }
   }
   return super.onTransact(code, data, reply, flags);
  }

  private static class Proxy implements com.test.vibratorbinder.IVibratorService {
   private android.os.IBinder mRemote;
   
   Proxy(android.os.IBinder remote) {
    mRemote = remote;
   }
   
   @Override
   public android.os.IBinder asBinder() {
    return mRemote;
   }

   public java.lang.String getInterfaceDescriptor() {
    return DESCRIPTOR;
   }

   @Override
   public boolean hasVibrator() throws android.os.RemoteException {
    android.os.Parcel _data = android.os.Parcel.obtain();
    android.os.Parcel _reply = android.os.Parcel.obtain();
    boolean _result;
    try {
     _data.writeInterfaceToken(DESCRIPTOR);
     mRemote.transact(Stub.TRANSACTION_hasVibrator, _data, _reply, 0);
     _reply.readException();
     _result = (0 != _reply.readInt());
    } finally {
     _reply.recycle();
     _data.recycle();
    }
    return _result;
   }

   @Override
   public void vibrate(int uid, java.lang.String opPkg, long milliseconds, int usageHint, android.os.IBinder token) throws android.os.RemoteException {
    android.os.Parcel _data = android.os.Parcel.obtain();
    android.os.Parcel _reply = android.os.Parcel.obtain();
    try {
     _data.writeInterfaceToken(DESCRIPTOR);
     _data.writeInt(uid);
     _data.writeString(opPkg);
     _data.writeLong(milliseconds);
     _data.writeInt(usageHint);
     _data.writeStrongBinder(token);
     mRemote.transact(Stub.TRANSACTION_vibrate, _data, _reply, 0);
     _reply.readException();
    } finally {
     _reply.recycle();
     _data.recycle();
    }
   }

   @Override
   public void vibratePattern(int uid, java.lang.String opPkg, long[] pattern, int repeat, int usageHint, android.os.IBinder token)
   throws android.os.RemoteException {
    android.os.Parcel _data = android.os.Parcel.obtain();
    android.os.Parcel _reply = android.os.Parcel.obtain();
    try {
     _data.writeInterfaceToken(DESCRIPTOR);
     _data.writeInt(uid);
     _data.writeString(opPkg);
     _data.writeLongArray(pattern);
     _data.writeInt(repeat);
     _data.writeInt(usageHint);
     _data.writeStrongBinder(token);
     mRemote.transact(Stub.TRANSACTION_vibratePattern, _data, _reply, 0);
     _reply.readException();
    } finally {
     _reply.recycle();
     _data.recycle();
    }
   }

   @Override
   public void cancelVibrate(android.os.IBinder token) throws android.os.RemoteException {
    android.os.Parcel _data = android.os.Parcel.obtain();
    android.os.Parcel _reply = android.os.Parcel.obtain();
    try {
     _data.writeInterfaceToken(DESCRIPTOR);
     _data.writeStrongBinder(token);
     mRemote.transact(Stub.TRANSACTION_cancelVibrate, _data, _reply, 0);
     _reply.readException();
    } finally {
     _reply.recycle();
     _data.recycle();
    }
   }
  }

  static final int TRANSACTION_hasVibrator = (android.os.IBinder.FIRST_CALL_TRANSACTION + 0);
  static final int TRANSACTION_vibrate = (android.os.IBinder.FIRST_CALL_TRANSACTION + 1);
  static final int TRANSACTION_vibratePattern = (android.os.IBinder.FIRST_CALL_TRANSACTION + 2);
  static final int TRANSACTION_cancelVibrate = (android.os.IBinder.FIRST_CALL_TRANSACTION + 3);
 }
 
 public boolean hasVibrator() throws android.os.RemoteException;
 public void vibrate(int uid, java.lang.String opPkg, long milliseconds, int usageHint, android.os.IBinder token) throws android.os.RemoteException;
 public void vibratePattern(int uid, java.lang.String opPkg, long[] pattern, int repeat, int usageHint, android.os.IBinder token) throws android.os.RemoteException;
 public void cancelVibrate(android.os.IBinder token) throws android.os.RemoteException;
}

    看來本體是一個 interface，我們先從 Proxy 類開始分析吧。會從這個類開始講不是沒有原因的，往裡面一看：每一個函式的內容，不就是 BpMyVibrator 裡面函式在做的事嗎，像一個代理人一樣，把傳進來的函式參數打包進 Parcel，調用 IBinder 的 transact() 傳送出去，瞧，連名字大部份都一樣呢！之前說 AIDL 一定會丟出例外，所以若要讀取返回值的話，要先把 exceptionCode 讀出來，在這邊也可以得到應證。

    再來看看 Stub 類吧。裡面最大的一部分，onTransact()，雖然前一篇沒有介紹，但您應該看得出來他就是接收端的處理函式，而我們在這邊要看的，是 asInterface()。韓式裡面似乎也很好懂，原理甚至命名都跟 IMPLEMENT_META_INTERFACE() 幾乎一模一樣：先看看自己有沒有實作對應的業務的介面，如果沒有的話，就 new 一個 Proxy，角色等同於前一篇的 BpMyVibrator，解讀上完全沒有障礙。

    透過剛剛的解析，就可以了解為什麼前一篇的 IMyVibrator 的那些 virtual 函式要長成那樣，然後 descriptor 要設定成那個值了。因為雖然我們要溝通的對方，VibratorService，是活在 system_server 裏的 Java Service，但多虧了 Binder，我們只要業務介面、descriptor 是一樣的，就可以跟他通訊。

Summary（謎之音：本篇重點似乎只是展示編譯完、排版好的 AIDL（笑））

    這篇文章希望透過將 Java 的 Binder API 與 Native 的 Binder API 互相呼應對照，讓您能夠對這個在 AOSP 中佔有重要地位的 IPC 系統有更透徹的了解，進而可以靈活的運用。

Using Binder to Vibrate - Android Binder 實戰 (Client Side, First Part)

    這次要示範的是如何使用 AOSP 裡面的 Binder 相關 API，來跟 system_server 裏的 VibratorService 通訊，使用震動器。

    我不是要解析 Binder 內部原理，解析 Android 原始碼的文章去 CSDN 隨便搜都一大堆(大陸人超喜歡解析原始碼)，本文後面附的參考書籍也不錯，我在這邊單純只是要示範如何使用而已。以下講解的 App，已經放上 Github 了：
https://github.com/mshockwave/android-binder-demo-with-vibrator

架構

    我本來的計劃是想直接載入 libhardware(或者 libhardware_legacy) 來操作 vibrator，但是想也知道：這種不用在 Manifest 聲明權限就可以使用硬體的好康，要是現在還存在的話 Google 早就不用玩了，process 本身權限也不夠。當然如果你有 root 權限的話，可以寫一個像是 MediaServer 一樣的 native service 來做前述的事。在不 root 的情況下，似乎就只剩下乖乖跟 system_server 進行通訊來使用 vibrator 了。
    因此這個 App 就是製作一個「Native 版 AIDL 檔」來跟 system_server 裏的 VibratorService 溝通。而後面也會講到，事實上 AIDL 檔只是 Binder 的 Java 綁定的一個包裝罷了。下列文中會用到一些 AOSP 的 header files 或是 shared library，礙於篇幅關係，就先不在這邊說明如何引入了。

（一）Binder 初識

    Binder 乍看之下還頗複雜的，但仔細一看才發現，看起來複雜的地方只是「包裝」而已，它核心的部分也是簡單的訊息傳遞。藉由哪邊傳遞訊息呢？答案是 kernel ，Binder 負責訊息傳遞的組件事實上是一個 kernel module，我剛開始非常好奇為什麼要搞那麼複雜，但後來有稍微理解了一點：kernel 的記憶體空間也是每個 process 共享的，跟 Sys V 的 shared memory IPC 道理一樣是共用記憶體。

    再來我們要有的概念就是 Binder 可以分成「業務」以及「通訊」兩部分。上面剛提到的，Binder 的包裝看起來很複雜，事實上他裡面就是把業務以及通訊融合在一起，傳統的 IPC (像是我們熟悉的 Sys V 系列) 都是要手動將資料做讀寫，而 Binder 則是將物件導向裡的介面以及繼承發揮得淋漓盡致，把資料的讀寫全部藏起來。

（二）業務介面：IInterface

    先來從最簡單的業務介面介紹起。(IMyVibrator.h)

class IMyVibrator : public IInterface {

public:
    DECLARE_META_INTERFACE(MyVibrator);

    virtual bool hasVibrator(void) = 0;

    virtual void vibrate(int32_t uid, 
                         String16& opPkg, 
                         int64_t milliseconds, int32_t usageHint, 
                         sp<IBinder>& token) = 0;

    virtual void vibratePattern(int32_t uid, 
                                String16& opPkg, 
                                int64_t pattern[], int32_t repeat, int32_t usageHint,
                                sp<IBinder>& token) = 0;

    virtual void cancelVibrate(sp<IBinder>& token) = 0;

};

請先忽略函式參數的意思。這邊定義的就是最終客戶端操作的介面，其中最有趣的莫過於 DECLARE_META_INTERFACE 這個 macro 了，他定義於 IInterface 裡，目前 5.0.0 (因為最近幾個版本 AOSP 檔案路徑變動很多) 座落在 frameworks/native/include/binder/IInterface.h。

#define DECLARE_META_INTERFACE(INTERFACE)                               \
    static const android::String16 descriptor;                          \
    static android::sp<I##INTERFACE> asInterface(                       \
            const android::sp<android::IBinder>& obj);                  \
    virtual const android::String16& getInterfaceDescriptor() const;    \
    I##INTERFACE();                                                     \
    virtual ~I##INTERFACE();                                            \

上方只是先單純的做一些宣告，descriptor 顧名思義就是用來辨識這個 Binder 節點，要注意的是不要跟後面講到 ServiceManager 裡 addService/getService 的那個服務名稱搞混，這邊的 descriptor 是 Binder 驅動裡的辨識符，而後者的服務名稱只適用於 ServiceManager。asInterface 則是要等一下才會講到，但如果有用過 AIDL 的朋友就會想起，使用 AIDL 時也有 asInterface 這個函式 ( 例：IMyInterface.Stub.asInterface() )，而我在這邊就先告訴您：他們兩者做的事是一樣的。

（三）客戶端通訊介面：BpInterface

剛剛的業務介面可能會讓你一頭霧水，感覺沒在做什麼，但沒錯，就如字面上的意思：他只是個介面，所以先請耐心地看下去，看看負責通訊的 BpInterface 類。BpInterface 我們在這邊要知道的，首先是他繼承了 BpRefBase。

template<typename INTERFACE>
class BpInterface : public INTERFACE, public BpRefBase
{
public:
                                BpInterface(const sp<IBinder>& remote);

protected:
    virtual IBinder*            onAsBinder();
};

而 BpRefBase 裡有一個成員變數 mRemote 以及跟他相關的 remote() 函式。

class BpRefBase : public virtual RefBase
{
protected:
                            BpRefBase(const sp<IBinder>& o);
    virtual                 ~BpRefBase();
    virtual void            onFirstRef();
    virtual void            onLastStrongRef(const void* id);
    virtual bool            onIncStrongAttempted(uint32_t flags, const void* id);

    inline  IBinder*        remote()                { return mRemote; }
    inline  IBinder*        remote() const          { return mRemote; }

private:
                            BpRefBase(const BpRefBase& o);
    BpRefBase&              operator=(const BpRefBase& o);

    IBinder* const          mRemote;
    RefBase::weakref_type*  mRefs;
    volatile int32_t        mState;
};

我們透過 BpInterface 的建構子來設定 mRemote，這點請稍微留意，到時候則會呼叫 mRemote 的 transact() 函式來傳送處理過後的訊息，而這邊也是本篇文章討論 Binder 部分的最底層，再往下如何跟 Binder 驅動溝通，就留給各位了，您也可以參見本篇文章後面附上的參考書籍，裡面有非常詳盡的解說。

（三）通訊 + 業務： BpXXXX

    到了這步驟，我們當然就要把業務以及通訊部分融合在一起啦。(MyVibratorProxy.cpp)

using namespace android;

//Client proxy interface
class BpMyVibrator : public BpInterface<IMyVibrator> {

public:
    BpMyVibrator(const sp<IBinder>& remote) : BpInterface<IMyVibrator>(remote) {
        ALOGD("BpMyVibrator constructor");
    }

    virtual bool hasVibrator(void) {
        ALOGD("BpMyVibrator::hasVibrator invoked");
        Parcel _data, _reply;
        _data.writeInterfaceToken(IMyVibrator::getInterfaceDescriptor());
        remote() -> transact(TRANSACTION_hasVibrator, _data, &_reply);
        /*
        * Must read the exception before fetching return data!!
        * Since methods in AIDL file will always throw RemoteException
        */
        ALOGD("%s exception code: %d", __func__, _reply.readExceptionCode());
        int retInt32 = _reply.readInt32();
        //ALOGD("Return in int: %d", retInt32);

        return (bool)(0 != retInt32);
    }

    virtual void vibrate(int32_t uid, 
                         String16& opPkg, 
                         int64_t milliseconds, int32_t usageHint, 
                         sp<IBinder>& token) {
        ALOGD("BpMyVibrator::vibrate invoked");
        Parcel _data, _reply;
        _data.writeInterfaceToken(IMyVibrator::getInterfaceDescriptor());
        _data.writeInt32(uid);
        _data.writeString16(opPkg);
        _data.writeInt64(milliseconds);
        _data.writeInt32(usageHint);
        _data.writeStrongBinder(token);

        remote() -> transact(TRANSACTION_vibrate, _data, &_reply);
        //ALOGD("%s exception code: %d", __func__, _reply.readExceptionCode());
    }

    virtual void vibratePattern(int32_t uid, 
                                String16& opPkg, 
                                int64_t pattern[], int32_t repeat, int32_t usageHint, 
                                sp<IBinder>& token) {
        ALOGD("BpMyVibrator::vibratePattern invoked");
        Parcel _data, _reply;
        _data.writeInterfaceToken(IMyVibrator::getInterfaceDescriptor());
        _data.writeInt32(uid);
        _data.writeString16(opPkg);
        size_t len = (size_t)(sizeof(pattern) / sizeof(pattern[0]));
        _data.write((const void*)pattern, len);
        _data.writeInt32(repeat);
        _data.writeInt32(usageHint);
        _data.writeStrongBinder(token);

        remote() -> transact(TRANSACTION_vibratePattern, _data, &_reply);
        //ALOGD("%s exception code: %d", __func__, _reply.readExceptionCode());
    }

    virtual void cancelVibrate(sp<IBinder>& token) {
        ALOGD("BpMyVibrator::cancelVibrate invoked");
        Parcel _data, _reply;
        _data.writeInterfaceToken(IMyVibrator::getInterfaceDescriptor());
        _data.writeStrongBinder(token);

        remote() -> transact(TRANSACTION_cancelVibrate, _data, &_reply);
        //ALOGD("%s exception code: %d", __func__, _reply.readExceptionCode());
    }
};

IMPLEMENT_META_INTERFACE(MyVibrator, "android.os.IVibratorService");

先從建構子開始看，前面有提到，我們是用建構子來傳遞通訊用的 IBinder 物件，所以一定要呼叫父建構子。接下來每一個函式都是我們在 IMyVibrator 裡宣告的業務函式，看看每一個函式都在做什麼？看起來就是把函式參數稍微處理一下，打包到 Parcel 型態的包裹裡，然後用前面提過的 remote()  與 transact() 函式傳送出去。到這邊您大概就猜的出來 BpInterface 的那個 "p" 是什麼的縮寫了：proxy。

    沒錯，在 BpMyVibrator 裡我們做的工作就像一個 proxy，代理人，當客戶端在使用 IMyVibrator 時，其實他們是呼叫這些代理函式。還有另一個問題就是，Parcel 有提供像是 int32, float 等基本型態的封裝，那麼較複雜的物件呢？答案是你的類別必須實作 Parcelable 介面，把類別可以變成 Parcel 攜帶，這部分 Android 官方文件介紹的很清楚(雖然是 java 的 Parcelable，但大同小異)，這邊就不提了。另外 transact() 函式第一個參數是用來辨識傳送給哪一個函式，要注意的是 AIDL 檔案在生成的時候，是照函式的順序由上而下編號的，例如我這次傳送的對象，IVibratorService，AIDL 檔 (位於 /frameworks/base/core/java/android/os/IVibratorService.aidl) 長這樣：

interface IVibratorService
{
    boolean hasVibrator();
    void vibrate(int uid, String opPkg, long milliseconds, int usageHint, IBinder token);
    void vibratePattern(int uid, String opPkg, in long[] pattern, int repeat, int usageHint, IBinder token);
    void cancelVibrate(IBinder token);
}

所以我在客戶端這邊的編號就要按照他的順序：(IMyVibrator.h)

#define TRANSACTION_hasVibrator (IBinder::FIRST_CALL_TRANSACTION + 0)
#define TRANSACTION_vibrate (IBinder::FIRST_CALL_TRANSACTION + 1)
#define TRANSACTION_vibratePattern  (IBinder::FIRST_CALL_TRANSACTION + 2)
#define TRANSACTION_cancelVibrate  (IBinder::FIRST_CALL_TRANSACTION + 3)

FIRST_CALL_TRANSACTION 預設的數值是 1。

    在代理函式實作步驟裡面，有幾個重點。第一，Parcel 包以什麼順序寫入，在另一端就會以什麼順序解開，Parcel 並沒有像 Bundle 那樣是以 key-value 對儲存。第二，您應該有發現，傳送出去的包，第一個寫入的一定是前面提到的 descriptor，以供 Binder 辨識對象。第三，Parcel 原生支援打包 IBinder 物件，也就是 writeStrongBinder() 函式 (應該說是打包 strong pointer 型態的 IBinder，也就是 sp<IBinder>)，因此可以看到 AOSP 裡常常把一個 IBinder 傳送給對方，當作單純的 token，就是不想再多創造一個 Parcelable 的物件，而多利用原生支持的 IBinder 傳送。第四，如上方程式碼註解的，如果對方有丟出 Exception，例如我們這個 App 因為對方是用 java 的 AIDL，一定會丟出 RemoteException，那我們必須先調用 readExceptionCode() 讀取例外，接著才能讀取到正確的返回值 (我那時在這邊卡很久)。

    在 MyVibratorProxy.cpp 最後一行有一個 IMPLEMENT_META_INTERFACE 的 macro，是不是很眼熟呢？沒錯，他就是 DECLARE_META_INTERFACE 的「另一半」，實作那些宣告。直接來看看他的內容：(也是座落於 IInterface.h)

#define IMPLEMENT_META_INTERFACE(INTERFACE, NAME)                       \
    const android::String16 I##INTERFACE::descriptor(NAME);             \
    const android::String16&                                            \
            I##INTERFACE::getInterfaceDescriptor() const {              \
        return I##INTERFACE::descriptor;                                \
    }                                                                   \
    android::sp<I##INTERFACE> I##INTERFACE::asInterface(                \
            const android::sp<android::IBinder>& obj)                   \
    {                                                                   \
        android::sp<I##INTERFACE> intr;                                 \
        if (obj != NULL) {                                              \
            intr = static_cast<I##INTERFACE*>(                          \
                obj->queryLocalInterface(                               \
                        I##INTERFACE::descriptor).get());               \
            if (intr == NULL) {                                         \
                intr = new Bp##INTERFACE(obj);                          \
            }                                                           \
        }                                                               \
        return intr;                                                    \
    }                                                                   \
    I##INTERFACE::I##INTERFACE() { }                                    \
    I##INTERFACE::~I##INTERFACE() { }                                   \

我們最困惑的 asInterface() 函式，終於在這邊獲得解答了。先看看第12行到第14行在做什麼吧，queryLocalInterface 宣告在 IBinder 類別裡，他的註解說：

/**
* Check if this IBinder implements the interface named by
* descriptor. If it does, the base pointer to it is returned,
* which you can safely static_cast<> to the concrete C++ interface.
*/

看來就是說，如果這個「業務 + 通訊 綜合體」有實作 descriptor 對應的業務介面，就把該業務介面回傳。但是 BpInterface 與 BnInterface 這兩者貌似只有 BnInterface 有實作這個函式，因此那三行現在對我們來說是沒用的，會回傳 NULL。再看下去，發現原來 asInterface 在這邊的作用單純只是 new 一個 BpInterface！那新的 BpInterface 怎麼通訊？想起前面說要稍微留意一下 BpInterface 的建構子嗎？看看這邊，obj 這個 IBinder 物件就是通訊用的，也就是會變成 BpInterface 中的 mRemote 值，可以透過 remote()  得到。這下您應該可以理解這小段的標題為什麼有一個 BpXXXX 了吧：一定要把 「業務 + 通訊 綜合體」命名成 BpXXXX 才行。

Summary

    Binder 美的地方就在於他把 IPC 向上提升一個境界，將物件導向的概念融合於自身，Firefox OS(B2G) 也有使用這個觀念，他們使用 local socket，但是用一些 IDL(介面描述語言) 加以包裝，讓開發者與使用者不用再直接處理通訊的事情。

    Second Part 將會分析比較 AIDL 與 此篇的原生 Binder 的實作，希望能夠更融會貫通，不管是 Native 對 Native 的通訊，還是如本 App 一樣 Native 對 Java 的通訊，都可以得心應手。

Book

《 深入理解 Android，卷 I 》(簡)，鄧凡平，機械工業出版社，2011。

Note: 作者是 Android Media 領域的專家，所以有關 MediaServer 等等的部分書裡都介紹的不錯，唯一可惜的是 HAL 層提到的比較少，然後作者有時候用的比喻台灣人可能比較難以理解（笑。

   

How To Use Android Internal APIs

    When I was working on this demo(exercise, actually), I want to use the ServiceManager java class(which is hidden in Android SDK) since the demo is aim to build a simple native service which has a role like the system_server ( accessing some crucial system parts like hardware )except that it isn't live in the system_server process. So neither the bindService routine, which is designed for "App Service", nor the getService method, which is created for the services live in system_server, will work.

    Here are the steps:

1. We need to build the AOSP, at least some specific framework parts, which located mostly in framework/base. You can go there and invoked the mma or mm command to build the module instead of the whole Android system.
      
2. Assume your output dir is out, then go to out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/, copy the classes.jar to ${YOUR_SDK_DIR}/platforms/android-${YOUR_API_LEVEL}/ .
      
3. In the dir ${YOUR_SDK_DIR}/platforms/android-${YOUR_API_LEVEL}, unpack both the classes.jar we've just copy and the android.jar since a jar file is simply an archive file.
      
4. This step depends on your need. Since I only wants the ServiceManager class, I only have to copy content in the android folder in the output of the classes.jar, which holds the android.* classes we normally seen in the java import statements, into the corresponding android folder in the output of the android.jar. By doing this, you can still keep some libraries like dalvik.* in the origin SDK but not exist in the AOSP framework part.
      
5. Change dir to the modified android.jar output dir. Repack them using the command:
   jar -cf ./your_new_android.jar *
   Remember to cascade your new android.jar file path RIGHT AFTER the -f option.
      
6. Then you will got the new android.jar which can used in the App development. I also kept the original android.jar and wrote a simple script to switch between the origin one and the "Hacked" one:
   

   #!/bin/bash
   
   USAGE="Usage: ${0} <aosp|sdk>"
   
   case "$1" in 
   "aosp") ln -sf android_aosp.jar android.jar
    ;;
   
   "sdk") ln -sf android_sdk_origin.jar android.jar
    ;;
   *) echo $USAGE
    ;;
   
   esac
   
   exit 0
   

    In the old Eclipse era, we can add the new android.jar to the build path and raise the priority among the dependency libraries. However, since Android Studio is highly integrated with Android SDK, we can't modify the actual SDK files in the IDE.