### Description

Allow an Airflow DAG to define a Databricks job with the `api/2.1/jobs/create` (or `api/2.1/jobs/reset`) endpoint then run that same job with the `api/2.1/jobs/run-now` endpoint. This would give similar capabilities as the DatabricksSubmitRun operator, but the `api/2.1/jobs/create` endpoint supports additional parameters that the `api/2.1/jobs/runs/submit` doesn't (e.g. `job_clusters`, `email_notifications`, etc.).

### Use case/motivation

Create and run a Databricks job all in the Airflow DAG. Currently, DatabricksSubmitRun operator uses the `api/2.1/jobs/runs/submit` endpoint which doesn't support all features and creates runs that aren't tied to a job in the Databricks UI. Also, DatabricksRunNow operator requires you to define the job either directly in the Databricks UI or through a separate CI/CD pipeline causing the headache of having to change code in multiple places.

### Related issues

_No response_

### Are you willing to submit a PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

"""
        Return the lifecycle state of the run.
        :param run_id: id of the run
        :return: string with lifecycle state
        """
        return self.get_run_state(run_id).life_cycle_state
    def get_run_state_result(self, run_id: int) -> str:
        """
        Return the resulting state of the run.
        :param run_id: id of the run
        :return: string with resulting state
        """
        return self.get_run_state(run_id).result_state
    def get_run_state_message(self, run_id: int) -> str:
        """
        Return the state message for the run.
        :param run_id: id of the run
        :return: string with state message
        """
        return self.get_run_state(run_id).state_message
    def get_run_output(self, run_id: int) -> dict:
        """
        Retrieve run output of the run.
        :param run_id: id of the run
        :return: output of the run
        """
        json = {"run_id": run_id}
        run_output = self._do_api_call(OUTPUT_RUNS_JOB_ENDPOINT, json)
        return run_output
    def cancel_run(self, run_id: int) -> None: