You can check whether a vApp is busy in two ways:
1) get the vApp, observe the <Tasks> element inside the vApp and check for any tasks in 'running' state: if there's a running task then the vApp is busy and cannot be recomposed
2) using the query service you can execute a vApp query (GET /api/query?type=adminVApp) to get all vApps (or filter by name to get the vApp you're interested in): inside the query results there's a boolean field called 'isBusy' showing whether the vApp is busy or not.
Regards,
Todor Todorov