Jekyll2021-10-24T18:39:45+01:00https://ayeshalshukri.co.uk/feed.xmlAyesh AlshukriResearch Software Engineer specialising in Personalised Healthcare, Precision Medicine and Clinical Trials <a href="/about">@OxfordUniversity</a>.Ayesh AlshukriSolution for Docker issue failed to solve: rpc error: code = unknown desc2021-10-18T00:00:00+01:002021-10-18T00:00:00+01:00https://ayeshalshukri.co.uk/category/guides/docker-error-failed-to-solve-with-frontend-dockerfile<figure>
<img src="DockerDeployment.png" alt="docker-error-failed-to-solve-with-frontend-dockerfile" />
</figure>
<p>This post presents a solution to an error I encountered recently with Docker
when trying to bring up a docker container using docker-compose.</p>
<blockquote>
<p>Skip to the <a href="#solution">solution that resolved this issue for me</a></p>
</blockquote>
<p>The solution in the post helped me fix
the Docker issue
<code class="language-plaintext highlighter-rouge">failed to solve: rpc error: code = unknown desc = failed to solve with frontend dockerfile v0</code></p>
<h1 id="docker-error-messages">Docker Error Messages</h1>
<p>If you have been having the same problem as me,
you might get one of the following error messages from Docker:</p>
<p><code class="language-plaintext highlighter-rouge">docker failed to solve rpc error code = unknown desc</code></p>
<p><code class="language-plaintext highlighter-rouge">docker failed to solve with frontend dockerfile.v0</code></p>
<p><code class="language-plaintext highlighter-rouge">failed to solve: rpc error: code = unknown desc = failed to solve with frontend dockerfile v0</code></p>
<p><code class="language-plaintext highlighter-rouge">Docker build error: failed to solve with the frontend dockerfile</code></p>
<h1 id="problem">Problem</h1>
<p>The problem with Docker was happening when I was trying to bring up docker
using a docker-compose file. I was using a command similar to this one:</p>
<p><code class="language-plaintext highlighter-rouge">> sudo -E docker-compose -f docker-compose.yml -f docker-compose.local.yml up --build</code></p>
<p>The error message I got was as follows:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ayesh@computer:$ sudo docker-compose -f docker-compose.local.yml up -d
[+] Building 0.0s (1/2)
=> [internal] load build definition from data 0.0s
=> => transferring dockerfile: 48.90kB 0.0s
failed to solve: rpc error: code = Unknown desc =
failed to solve with frontend dockerfile.v0: failed to read dockerfile:
read /var/lib/docker/tmp/buildkit-mount939660890/data-extraction:
is a directory
</code></pre></div></div>
<p>I tried a few variations to bring up the docker containers</p>
<p><code class="language-plaintext highlighter-rouge">> sudo docker-compose -f docker-compose.local.yml up -d</code></p>
<p>I got similar error messages from docker:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ayesh@computer:$ sudo docker-compose -f docker-compose.local.yml up -d
[+] Building 0.6s (2/2) FINISHED
=> [internal] load build definition from data 0.6s
=> => transferring dockerfile: 59.91MB 0.6s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
failed to solve: rpc error: code = Unknown desc =
failed to solve with frontend dockerfile.v0: failed to read dockerfile:
read /var/lib/docker/tmp/buildkit-mount070889300/data-extraction:
is a directory
</code></pre></div></div>
<p>I also tried to clear out existing docker containers and images from the cache.
<code class="language-plaintext highlighter-rouge">> sudo docker system prune -a -f</code>
This did not work when I tried to rebuild docker containers as above.</p>
<p>The consistent error from each of the commands I tried that failed seemed to be
not finding a docker file.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>failed to solve:
rpc error: code = Unknown desc =
failed to solve with frontend dockerfile.v0:
failed to read dockerfile:
read /var/lib/docker/tmp/buildkit-mount070889300/data:
is a directory
</code></pre></div></div>
<h1 id="docker-setup">Docker Setup</h1>
<p>You can <a href="#solution">skip to the solution here</a>, as
this section helps check if docker is correctly installed.</p>
<p>The first step you can try is to make sure you can actually run a ‘hello world’
docker image successfully.
You can do this by executing the following command:</p>
<p><code class="language-plaintext highlighter-rouge">> docker run hello-world</code></p>
<p>The output of this command should be as follows:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ayesh@computer:$ sudo docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
2db29710123e: Pull complete
Digest: sha256:9ade9cc2e26189a19c2e8854b9c8f1e14829b51c55a630ee675a5
Status: Downloaded newer image for hello-world:latest
Hello from Docker!
This message shows that your
installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the
Docker Hub. (amd64)
3. The Docker daemon created a new container from that image which
runs the executable that produces the output you are
currently reading.
4. The Docker daemon streamed that output to the Docker client,
which sent it to your terminal.
To try something more ambitious,
you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/get-started/
</code></pre></div></div>
<p>As suggested in the output of the docker hello world example,
you can try a more advanced docker test, which would be to run
the latest ubuntu image and execute command via the
bash terminal in ubuntu.</p>
<p><code class="language-plaintext highlighter-rouge">> docker run -it ubuntu bash</code></p>
<p>The output should be as follow, and should give you
terminal access as the root user.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ayesh@computer:~$ sudo docker run -it ubuntu bash
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
7b1a6ab2e44d: Pull complete
Digest: sha256:626ffe58f6e7566e00254b638eb7e0f3b
Status: Downloaded newer image for ubuntu:latest
root@caf8018236b0:/#
</code></pre></div></div>
<h1 id="solution">Solution</h1>
<p>The solution is to explicitly give the name of the docker file in the docker-compose.yml.
Change how you specify the location of your dockerfile from
<code class="language-plaintext highlighter-rouge">dockerfile: .</code>
to
<code class="language-plaintext highlighter-rouge">dockerfile: Dockerfile</code></p>
<p>This is an sample of the docker-compose file which did not work on Ubuntu 20.04 or Windows10,
it only seems to work on MacOS only.
As you can see, the dockerfile location is specified as a ‘.’</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>version: "3"
...
build:
context: ../.
dockerfile: .
</code></pre></div></div>
<p>The sample docker-compose file shown below is a working
cross platform file, which works on MacOS, Ubuntu, Windows10.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>version: "3"
...
build:
context: ../.
dockerfile: Dockerfile
</code></pre></div></div>
<p>As you can see, the line specifying the dockerfile has been changed
from a dot ‘.’ to the explicit file name, in my case, it was ‘Dockerfile’.</p>
<p>This also works with lower and uppercase docker files as
<code class="language-plaintext highlighter-rouge">dockerfile: Dockerfile</code> or <code class="language-plaintext highlighter-rouge">dockerfile: dockerfile</code></p>
<h1 id="conclusion">Conclusion</h1>
<p>The problem seemed to originate from the
cross platform development and testing of the docker-compose script,
between MacOS and Ubuntu / Window10 using version 3 of docker-compose.
In the scenario of specifying the location of the dockerfile.</p>
<p>The original docker-compose file was built on MacOS,
I was trying to use the same file but on Ubuntu 20.04.
I believe this same problem can also occur if also trying to use docker-compose
developed using MacOS on Windows10.</p>
<p>If you are using MacOS, then this docker-compose file will most likely run
without any problems (if you do not have any other unrelated problems).</p>
<p>You can read more about how I have used
<a id="docker in other projects" href="/tags/#docker">docker in my other projects</a>.</p>Ayesh AlshukriGuide to automated deployment2021-07-16T00:00:00+01:002021-07-16T00:00:00+01:00https://ayeshalshukri.co.uk/category/guides/automated-deployment<p>This article presents a guide to automated scripted deployment for a non-technical audience.
It covers topics around software deployments, code deploy and the entire automated deployment pipeline.
If you are interested in the technical details,
this article also contains links to scripts you can try out.</p>
<blockquote>
<p>skip to <a href="#deployment-environment-terminology">Deployment Environment Terminology</a> or <a href="#deployment-process">Deployment Process</a></p>
</blockquote>
<p>The topics this article covers:</p>
<ul>
<li>What does deployment mean?</li>
<li>Is automated deployment appropriate for clinical and healthcare systems?</li>
<li>What are the benefits of deployment automation?</li>
<li>Issues that Automated Deployment Solves</li>
<li>Deployment Environment Terminology</li>
<li>Deployment Process</li>
<li>Deployment Script</li>
<li>Deployment Trigger</li>
</ul>
<h1 id="what-does-deployment-mean">What does deployment mean?</h1>
<p>What does deployment mean in IT/ software dev?
What is deploying software? What does it even mean?</p>
<p>Deploying code is a fancy term for updating code.
Nothing too complicated.
If we say, we are deploying the latest version of our code,
then we are simply updating the current code, to the latest version of the code.</p>
<p>Automating the ‘deployment’ of code, simply means we are updating our code, using an automated technique.
In this case, we often mean running a script.
The script, will often times, contains commands that would be run manually by a user,
if deploying (updating code) by hand.</p>
<p>An automated deployment is what enables you to deploy your software
to test, stage and production environments with the flick of a switch.
Automation is an essential part of the effective operation of software delivery life cycle.</p>
<h1 id="is-automated-deployment-appropriate-for-clinical-and-healthcare-systems">Is automated deployment appropriate for clinical and healthcare systems?</h1>
<p>According to MHRA / GCP, appropriate controls of the system are in place throughout the systems lifetime.
The system is fit for purpose and performs reliably and consistently as intended.
Computer systems should be periodically evaluated to confirm that they remain in a
valid state and are compliant.
This should include:</p>
<ul>
<li>current range of functionality</li>
<li>deviations records</li>
<li>incidents</li>
<li>problems</li>
<li><strong>upgrade history</strong></li>
<li>performance</li>
<li>reliability</li>
<li>security</li>
<li>validation</li>
</ul>
<p>[14.5.1 Validation Principles. MHRA Grey Guide]
[EudraLex The Rules Governing Medicinal Products in the European Union Volume 4
Good Manufacturing Practice Medicinal Products for Human and Veterinary Use Annex 11:
Computerised Systems – Equivalent to FDA 21CFR11]</p>
<p>Upgrade history is the key item from this list that automated deployments
aims to provide a compliant solution in-line with regulatory restrictions and guidelines.</p>
<h1 id="what-are-the-benefits-of-deployment-automation">What are the benefits of deployment automation?</h1>
<p>This section covers the numerous benefits of automated deployment.
If you have any others, please reach out and <a href="/contact/">get in touch</a>.</p>
<p>The main benefits come from
reducing the possibility of errors because manual deployment is error-prone.
Any process involving human input introduces the possibility of errors and defect in the final product.
Repetitive manual deployment tasks can harm employee morale.
Computer are good at repetitive tasks, human are good a creativity, like solving code problems.
Saving time as manual deployment is very time-consuming.
Developers within the team can otherwise spend their time working on improving the application.
Consistency and repeatability, once and automated deployment is configured,
then 99.9% of the time (famous last words!!) the process is the same every time.
This process contributes to more frequent and stable releases,
and produces high consistency and greatly assists in diagnosis of errors as it
reduces the possible variables, like human mistakes.</p>
<p>In terms of audit and compliance:</p>
<ul>
<li>Reproducible</li>
<li>Traceable</li>
<li>Evidenced</li>
<li>Transparent</li>
<li>Maintain separated concerns (devs don’t need server login).</li>
<li>Reduces defects (caused by human error)</li>
<li>Anyone can deploy</li>
<li>Audit Trail</li>
</ul>
<p>In terms of software quality improvement:</p>
<ul>
<li>More frequent releases</li>
<li>Immediate feedback (deployment is faster)</li>
<li>Smaller code changes are simpler</li>
<li>Reduced errors</li>
<li>Fault isolation is simpler</li>
<li>Increased productivity</li>
<li>Faster and more efficient deployment</li>
</ul>
<h1 id="issues-that-automated-deployment-solves">Issues that Automated Deployment Solves</h1>
<p>This section lists some of the issues that automated deployment solves:</p>
<ul>
<li>Increased cost of producing software</li>
<li>Poor Code Quality</li>
<li>Meaningless Work</li>
<li>Time wasting</li>
<li>Developers vs. Server Admin wars</li>
<li>No runtime version management</li>
<li>Long Feedback Loops</li>
<li><a href="/contact/">any more?…</a></li>
</ul>
<h1 id="deployment-environment-terminology">Deployment Environment Terminology</h1>
<p>It is important, before we cover the important aspects of automated deployment, we understand certain terminology in the deployment chain.</p>
<p>I am mainly going to focus on environment terminology.
There is a lot of other terminology, but it is not relevant in this particular context.</p>
<h4 id="what-is-an-environment">What is an environment?</h4>
<p>An environment is a term used for a ‘place’ where code resides.
The type of environment, dictates (or should dictate) the state of the code, when it is in this ‘place’. The term environment, is sometimes shortened to ‘env’.</p>
<p>Example;
A ‘play’ environment, we would expect unstable code, and any changes are not going to dangerous/ detrimental.
Where as;
A ‘real’ environment, we would expect stable code, and any changes to have real world consequences.</p>
<h4 id="deployment-environments">Deployment Environments</h4>
<ul>
<li>Dev / local
This is our laptops/ desktops/ work stations
<ul>
<li>Used by developers</li>
<li>No real data (seed data)</li>
</ul>
</li>
<li>Test
Server used for multi user testing
<ul>
<li>Used by developers and QA</li>
<li>No real data (seed data)</li>
</ul>
</li>
<li>Stage
Known as pre-production. The place we get to do any final tests, before
deployment to a ‘real world’ production environment.
<ul>
<li>Used by developers, QA and End Users.</li>
<li>Sanitized/limited real data</li>
</ul>
</li>
<li>Production
Stable code, used by ‘real world’ end users.
<ul>
<li>production-test</li>
<li>production-live</li>
</ul>
</li>
</ul>
<p>Now that we have the deployment environment terminology out of the way,
we can cover two the deployment process.</p>
<figure>
<img src="img/deployment-env-workflow.png" alt="deployment-env-workflow" />
</figure>
<!-- deployment-env-workflow-full.png -->
<h1 id="deployment-process">Deployment Process</h1>
<p>In this section, I will cover the deployment process.
There are two important aspects of the automated deployment process.
The first, is the deployment script.
The second is the deployment trigger.
This section will explain both these aspects which will be aimed at a mixed audience, I will not cover in-depth technical details.
For more in-depth detail you can read more about the
<a href="/category/guides/bash-shell-script-for-deploying-your-web-app/">deployment script</a>
and
<a href="/category/guides/automated-deployment-using-github-actions-ssh-and-shell-script/">deployment trigger</a>
in the respective technical articles.</p>
<h2 id="deployment-script">Deployment Script</h2>
<p>In this section I aim to cover the main aspects of the deployment script.
This will be in such a way that is accessible to a mix audience.
If you are interested in the technical details, I have a more in-depth article on this
<a href="/category/guides/bash-shell-script-for-deploying-your-web-app/">
deployment script, which you can read more about</a>.</p>
<p>The part of the deployment script we are interested in, is the main function.
You don’t have to understand bash/ shell programming to follow along.</p>
<p>The main function detects the environment the script is being run on.
The four types of environment are:</p>
<ul>
<li>Dev / Local</li>
<li>Test</li>
<li>Stage</li>
<li>Prod</li>
</ul>
<p>I will explain the deployment for each of these environments in turn.</p>
<h4 id="dev--local-env">Dev / Local Env</h4>
<p>The dev / local deployment script is used for testing only.
This is the environment that is used on your local laptop / desktop.
For technical audience; I actually mount the deploy directory and execute
this script on my docker container and test the deployment script works as expected.
For non technical audience, all you need to know, is this environment is used only for testing.</p>
<p>I will move on to explain the test env, as this is a good point to explain what is happening.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> if [ $env = "local" ]; then
echo "-local (dev)"
func_updateCode;
func_buildDev;
func_dataSeed;
func_updateLive;
func_clearCache;
</code></pre></div></div>
<h4 id="test-env">Test Env</h4>
<p>The block of code shown below presents what happens if the test env is detected.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> elif [ $env = "test" ]; then
echo "-test"
func_updateCode;
func_buildDev;
func_maintenanceMode true;
func_dataSeed;
func_updateLive;
func_clearCache;
func_maintenanceMode false;
</code></pre></div></div>
<p>The first function that is run is <code class="language-plaintext highlighter-rouge">updateCode</code>. This goes away, and grabs the latest code from github,
downloads to a temporary directory.
The key here, is downloading to a temporary directory, not a live one that is being served to users.</p>
<p>The next function is <code class="language-plaintext highlighter-rouge">buildDev</code>. This function does anything that needs to be done for the
application to work correctly. This can be tasks like download library, check settings and
create css and js.
The key here, is that we use the <code class="language-plaintext highlighter-rouge">buildDev</code> function, which means we build for a development application.
In reality, this means we would not compress css or js, and we install any library and
supporting applications like performance monitors or profilers, that will help with debugging.</p>
<p>Once we have grabbed the latest code, built our application, we then turn the current application into maintenance mode. What this does will differ depending on your application, but, in essence. It will display a message to users, saying something like “application going down for maintenance”.</p>
<p>Depending on your application, you can use this function to display a timer to users to save and logout. You can then sleep this script, or wait, until the specified time is over, and then proceed.</p>
<p>Once maintenance mode is enabled, I then run a function called <code class="language-plaintext highlighter-rouge">dataSeed</code> which deletes the entire test database, and restored all tables from scratch, and then populates with some seeding data. This seeding data contains a standard set of users with fixed passwords (Adam Admin and the gang). The seeding also populated with application data, so that users will see information displayed, and can interact/test the application.</p>
<p>Once we have completed the database part of the deployment,
we then move on to updating the code. We run the <code class="language-plaintext highlighter-rouge">updateLive</code> function to do this.
The code we downloaded from github, and then built, is copied to the main directory that is served to users.
We use <code class="language-plaintext highlighter-rouge">rsync</code> to only copy changes, and speed up the process.</p>
<p>Once the new files are copied, we then clear any cache and storage that might contain information on the old application. This is done by running the <code class="language-plaintext highlighter-rouge">clearCache</code> function.
Once this is done, we can then disable maintenance mode, and bring up the application.</p>
<h4 id="stage-env">Stage Env</h4>
<p>The block of code below shows the set of instructions running on the staging environment.
You will notice, most of the command are the same as the test environment.
The main difference is <code class="language-plaintext highlighter-rouge">buildProd</code> and <code class="language-plaintext highlighter-rouge">dataRestore</code> functions.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> elif [ $env = "stage" ]; then
echo "-stage"
func_updateCode;
func_buildProd;
func_maintenanceMode true;
func_dataRestore;
func_updateLive;
func_clearCache;
func_maintenanceMode false;
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">buildProd</code> command is largely similar to the <code class="language-plaintext highlighter-rouge">buildDev</code> command explained above, but this time, we build the application for the production setting. This means compressed css and js, along side restricting libraries to production only. Settings will also be configured for production, this is most likely will be non verbose output for errors etc.</p>
<p>The <code class="language-plaintext highlighter-rouge">dataRestore</code> function replaces the <code class="language-plaintext highlighter-rouge">dataSeed</code> function, but does, largely, configured the same aspect, which is the database.
As explained above, the <code class="language-plaintext highlighter-rouge">dataSeed</code> effectively deletes the database, and then populates with seeding data.
In contract, the <code class="language-plaintext highlighter-rouge">dataRestore</code> that is used in the stage env, restores a sanitized production database.
This means that testing can highlight problems/issues around edge cases from real world usage.</p>
<p>The remainder of the script is the same as I have previously explained above.</p>
<h4 id="production-env">Production Env</h4>
<p>The block of code below shows the set of instructions running on the production environment.
The instructions are similar to the staging, as the staging is trying to replication production. Staging in known as pre-production.
What is different for the production instructions, is a backup function <code class="language-plaintext highlighter-rouge">backupLive</code>. This backups up the production code and database before we copy new files.
You will also notice, we do not do any database changes, as this is a production machine, so we don’t want to mess with the data.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> elif [ $env = "prod" ]; then
echo "-prod"
func_updateCode;
func_buildProd;
func_maintenanceMode true;
func_backupLive;
func_updateLive;
func_maintenanceMode false;
</code></pre></div></div>
<h4 id="main-function">Main Function</h4>
<p>The full function containing a set of all of the above deployment environments is shown below.
If you want more technical details, they can be found in this
<a href="/category/guides/bash-shell-script-for-deploying-your-web-app/">technical article on bash script for deployment</a></p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># Function main()
# Detects env and deploys accordingly.
func_main(){
echo "-Main"
if [ $env = "local" ]; then
echo "-local (dev)"
func_updateCode;
func_buildDev;
func_dataSeed;
func_updateLive;
func_clearCache;
elif [ $env = "test" ]; then
echo "-test"
func_updateCode;
func_buildDev;
func_maintenanceMode true;
func_dataSeed;
func_updateLive;
func_clearCache;
func_maintenanceMode false;
elif [ $env = "stage" ]; then
echo "-stage"
func_updateCode;
func_buildProd;
func_maintenanceMode true;
func_dataRestore;
func_updateLive;
func_clearCache;
func_maintenanceMode false;
elif [ $env = "prod" ]; then
echo "-prod"
func_updateCode;
func_buildProd;
func_maintenanceMode true;
func_backupLive;
func_updateLive;
func_maintenanceMode false;
else
echo "Failure: ENV not set" >&2
echo "Exiting!!!" >&2
exit 1
fi
echo "-Done"
}
</code></pre></div></div>
<h2 id="deployment-trigger">Deployment Trigger</h2>
<p>In this section I aim to cover the main aspects of the deployment trigger.
This will be in such a way that is accessible to a mix audience.
If you are interested in the technical details, I have a more in-depth article on this
<a href="/category/guides/automated-deployment-using-github-actions-ssh-and-shell-script/">deployment trigger, which you can read more about</a>.</p>
<p>The deployment trigger is the task of running our deployment script.
In the setting I will explain here, we use github actions to trigger our deployment script.</p>
<p>Lets first discuss what a basic github action does,
and how this can trigger our deployment script.</p>
<p>As you can see from this github actions script, it has a name of ‘deploy mybranch’.
It is triggered when there is a push to branch ‘mybranch’.
Effectively, when there is a change to the code on ‘mybranch’, this script will be triggered.
The jobs section, will run any jobs you want. In this script, you can see,
I have simply stubbed a build, test, and deploy section.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>name: deploy mybranch
on:
push:
branches:
- mybranch
jobs:
# Build
# Test
# Deploy
</code></pre></div></div>
<p>The build and test sections would be used to first build the application, and then run the automated unit, integration, UAT tests. This happens before we run the deployment, if any problems with build or test, we would not progress to deployment.</p>
<p>As you can see from the code block below, it shows how we can use a github action to trigger our deployment script.
This github action, is triggered on push/change to ‘mybranch’, the deploy section, then connect to our server, and triggers the deploy script (as covered in the previous section above).</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>name: deploy mybranch
on:
push:
branches:
- mybranch
jobs:
# Build
# Test
# Deploy
- name: Install SSH Key
uses: shimataro/ssh-key-action@v2.1.0
with:
key: $
known_hosts: $
- name: Execute deployment Script
run: ssh username@myserver.com "cd application/$-Deploy && pwd && ./deploy.sh"
</code></pre></div></div>
<h2 id="env-triggers">Env Triggers</h2>
<p>We actually have github action triggers setup for our test, stage and production deployment environments.</p>
<p>The trigger script for test and stage is the same. Only difference, is the server that the deployment script is triggered on.</p>
<h4 id="test-env-trigger">Test Env Trigger</h4>
<p>Here is the github script for the test environment.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>name: deploy svr-test
on:
push:
branches:
- svr-test
jobs:
# Build
# Test
# Deploy
- name: Install SSH Key
uses: shimataro/ssh-key-action@v2.1.0
with:
key: $
known_hosts: $
- name: Execute deployment Script
run: ssh username@myserver.svr-test.com "cd application/$-Deploy && pwd && ./deploy.sh"
</code></pre></div></div>
<h4 id="stage-env-trigger">Stage Env Trigger</h4>
<p>Here is the github action script for the stage environment.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>name: deploy svr-stage
on:
push:
branches:
- svr-stage
jobs:
# Build
# Test
# Deploy
- name: Install SSH Key
uses: shimataro/ssh-key-action@v2.1.0
with:
key: $
known_hosts: $
- name: Execute deployment Script
run: ssh username@myserver.svr-stage.com "cd application/$-Deploy && pwd && ./deploy.sh"
</code></pre></div></div>
<h4 id="production-env-trigger">Production Env Trigger</h4>
<p>The github action trigger script for the production environment is slightly different to the one for test and stage.</p>
<p>First thing, is the production action is triggered when we cut a new release on the <code class="language-plaintext highlighter-rouge">svr-prod</code> branch.
This means, we first update code on the svr-prod branch, we then create a new release, once completed, this will kick off the github action.</p>
<p>The second thing, is the version bump script. This part of the script will increment the version of the production application automatically. It pulls the version number from the latest release in github, writes to a file. This can then be displayed within the application.</p>
<p>The final aspect, is that we deploy to both the production test (prod-test) and the production live (prod-live) within the last part of this script.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>name: deploy svr-prod
on:
release:
types: [published]
jobs:
# Test
# Build
# Bump version
# After successful test and build, then bump version before deployment.
version-bump:
runs-on: ubuntu-latest
name: Bump version
steps:
- name: Checkout
uses: actions/checkout@v2
with:
ref: svr-prod
- name: Update version and commit
run: |
date > code/latest-release.txt
targetConfigFile="config/version.txt"
git fetch --prune --unshallow
oldVersion=$(awk '{print $6}' $targetConfigFile)
newVersion="'"$(git describe --tags --abbrev=0 --always)"'"
echo "Bumping..."
echo " Old Version = "$oldVersion
echo " New Version = "$newVersion
awk '{$6 = "'$newVersion'"; print}' $targetConfigFile > $targetConfigFile".tmp" \
&& mv $targetConfigFile".tmp" $targetConfigFile
echo "Done."
git config user.name github-actions
git config user.email github-actions@github.com
git add .
git commit -m "Robot Commit: Auto version bump on svr-prod deploy"
git push
# Deploy
deploy:
needs: version-bump
runs-on: ubuntu-18.04
steps:
- name: Install SSH Key
uses: shimataro/ssh-key-action@v2.1.0
with:
key: $
known_hosts: $
- name: Execute deployment Script
run: ssh username@myserver.svr-prod.com
"cd application-prod-test/$-Deploy
&& ./deploy.sh"
run: ssh username@myserver.svr-prod.com
"cd application-prod-live/$-Deploy
&& ./deploy.sh"
</code></pre></div></div>
<h2 id="triggering-github-action">Triggering Github Action</h2>
<p>In the previous section, I explained how we trigger a github action.
We use triggers on push, which runs when there is a change on a specific branch
(svr-test, svr-stage). We also use on publish,
which triggers when there is a new release cut on a specific branch (svr-prod).</p>
<figure>
<img src="img/github-action.gif" alt="github action" />
</figure>
<p>Once an action has run, we can see if the action has completed successfully, or has failed.</p>
<figure>
<img src="img/action-list.png" alt="github action list" />
</figure>
<figure>
<img src="img/action-details.png" alt="github action details" />
</figure>
<figure>
<img src="img/action-details.gif" alt="github action details (animated)" />
</figure>
<p>The output of the deployment script is stored within github, as the trigger stays connected via ssh to the server, and stores all output.
We also store the output on the server itself.
Both these log files are perfect for traceability and audit.</p>Ayesh AlshukriThis article presents a guide to automated scripted deployment for a non-technical audience. It covers topics around software deployments, code deploy and the entire automated deployment pipeline. If you are interested in the technical details, this article also contains links to scripts you can try out.Laravel Jetstream Tutorial: How to create new pages and components2021-07-12T00:00:00+01:002021-07-12T00:00:00+01:00https://ayeshalshukri.co.uk/category/dev/laravel-jetstream-tutorial-create-new-pages<figure>
<img src="laravel.png" alt="main page image" />
</figure>
<p>This article presents a guide for customising your basic Laravel Jetstream installation.
In particular, this tutorial covers the questions like
How do you create a new page in Jetstream?
How do you build a Jetstream component?</p>
<p>If you are unsure what Laravel is, you can read more about
<a href="/category/dev/php-laravel-framework/">Laravel here</a>.
If you want to get a basic Laravel Jetstream application up and running,
so you can follow this
<a href="/category/guides/development-environment-docker-lemp-laravel-jetstream/">
tutorial, you can read this guide here</a>.</p>
<!--
How do you build a Jetstream component?
laravel jetstream create component
laravel jetstream tutorial: create new page
Laravel Jetstream tutorial
jetstream create component
laravel livewire examples
laravel livewire tutorial
laravel livewire tutorial pdf
-->
<h1 id="how-do-you-build-a-jetstream-component">How do you build a Jetstream component?</h1>
<p>To modify you need to know</p>
<ul>
<li>laravel blade components.</li>
<li>tailwind css</li>
</ul>
<p>What is the process to add a new page to Laravel Jetstream?
We are going to add a component to the application.</p>
<p>The basic process is to execute the command below to create the two files you need.</p>
<p><code class="language-plaintext highlighter-rouge">> php artisan make:component MyComponent</code></p>
<p>Executing this command, will create two files for you.</p>
<p><code class="language-plaintext highlighter-rouge">www/app/View/Components/MyComponent.php</code>
<code class="language-plaintext highlighter-rouge">www/resources/views/components/my-component.blade.php</code></p>
<p>If you open the <code class="language-plaintext highlighter-rouge">my-component.blade.php</code> you will see something like below.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><div>
<!-- Knowing is not enough; we must apply.
Being willing is not enough; we must do.
- Leonardo da Vinci -->
</div>
</code></pre></div></div>
<p>Uncomment this quote, or write your own text.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><div>
Knowing is not enough;
we must apply.
Being willing is not enough;
we must do.
- Leonardo da Vinci
</div>
</code></pre></div></div>
<p>Now you are free to use you new component by inserting the tag <code class="language-plaintext highlighter-rouge"><x-my-component></x-my-component></code>.</p>
<p>You can try this out by simple inserting this tag into the welcome or dashboard blades.</p>
<p><code class="language-plaintext highlighter-rouge">/resources/views/welcome.blade.php</code></p>
<p><code class="language-plaintext highlighter-rouge">/resources/views/dashboard.blade.php</code></p>
<p>How do we create a new page with link on the menu?
I will create a help page to show how I did this.</p>
<h1 id="how-do-add-a-new-page-to-laravel-jetstream">How do add a new page to Laravel Jetstream</h1>
<p>We are going to add a help page to the application.
This will be done in three steps.</p>
<ol>
<li>Create the route</li>
<li>Create the view blade</li>
<li>Add link to menu</li>
</ol>
<h2 id="create-the-route">Create the route</h2>
<p>Open the file <code class="language-plaintext highlighter-rouge">/routes/web.php</code>.
We want to add a block of code to direct us to the newly created help page.</p>
<p>To test if the route is working correctly,
you can simply return the words ‘help’ using this command <code class="language-plaintext highlighter-rouge">return 'help';</code></p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Route::middleware(['auth:sanctum', 'verified'])
->get('/help', function () {
return 'help';
})->name('help');
</code></pre></div></div>
<p>Once you created your route, if you get a 404, you might need to
issue this command to clear the routes.
<code class="language-plaintext highlighter-rouge">> php artisan route:clear</code></p>
<p>You can navigate to <code class="language-plaintext highlighter-rouge">http://localhost:8080/help</code> (if using my docker setup)
to make sure everything is working ok.</p>
<p>Once you are happy, this is the final block of code to use,
which will return the help view.
(we simply swapped <code class="language-plaintext highlighter-rouge">return 'help';</code> for <code class="language-plaintext highlighter-rouge">return view('help');</code>)</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Route::middleware(['auth:sanctum', 'verified'])->get('/help', function () {
return view('help');
})->name('help');
</code></pre></div></div>
<h2 id="create-the-view-blade">Create the view blade</h2>
<p>In this example, we want to create a single view,
that has two components.
The components we want to create on our help page, will be
‘guide’ and ‘support’.</p>
<p>We execute the command below to create the two files you need for the ‘guide’ component.</p>
<p><code class="language-plaintext highlighter-rouge">> php artisan make:component Guide</code></p>
<p>Executing this command, will create two files for you.</p>
<p>/app/View/Components/Guide.php
/resources/views/components/guide.blade.php</p>
<p>The component file, will look like this
/app/View/Components/Guide.php</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><?php
namespace App\View\Components;
use Illuminate\View\Component;
class Help extends Component
{
/**
* Create a new component instance.
*
* @return void
*/
public function __construct()
{
//
}
/**
* Get the view / contents that represent the component.
*
* @return \Illuminate\Contracts\View\View|\Closure|string
*/
public function render()
{
return view('components.guide');
}
}
</code></pre></div></div>
<p>The blade file, will look something like this
/resources/views/components/guide.blade.php</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><div>
<!-- Knowing is not enough; we must apply. Being willing is not enough; we must do. - Leonardo da Vinci -->
</div>
</code></pre></div></div>
<p>You can modify this file <code class="language-plaintext highlighter-rouge">guide.blade.php</code> to display some text.
Simply removing the comments from the quote is enough at this stage.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><div>
Knowing is not enough; we must apply. Being willing is not enough; we must do. - Leonardo da Vinci
</div>
</code></pre></div></div>
<p>Repeat this process for the support component.</p>
<p><code class="language-plaintext highlighter-rouge">> php artisan make:component Support</code></p>
<p>Executing this command, will create two files for you, as above.</p>
<p>/app/View/Components/Support.php
/resources/views/components/support.blade.php</p>
<p>Again, the only one we want to modify at this stage is the <code class="language-plaintext highlighter-rouge">support.blade.php</code> located in folder <code class="language-plaintext highlighter-rouge">/resources/views/components/</code>.
The file can be something like below:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><div>
An unexamined life is not worth living. - Socrates
</div>
</code></pre></div></div>
<p>Now we have our two components, guide, and support.
We want to create our help page, which will contain there two components.</p>
<p>Create a file <code class="language-plaintext highlighter-rouge">/resources/views/help.blade.php</code></p>
<p>Add the content below</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><x-app-layout>
<x-slot name="header">
<h2 class="font-semibold text-xl text-gray-800 leading-tight">
</h2>
</x-slot>
<div class="py-12">
<div class="max-w-7xl mx-auto sm:px-6 lg:px-8">
<div class="bg-white overflow-hidden shadow-xl sm:rounded-lg">
<x-guide />
</div>
</div>
</div>
<div class="py-12">
<div class="max-w-7xl mx-auto sm:px-6 lg:px-8">
<div class="bg-white overflow-hidden shadow-xl sm:rounded-lg">
<x-support />
</div>
</div>
</div>
</x-app-layout>
</code></pre></div></div>
<p>you will notice, in our help blade, we are calling two components.
The guide component is called using the line <code class="language-plaintext highlighter-rouge"><x-guide /></code>.
The support component is called using the line <code class="language-plaintext highlighter-rouge"><x-support /></code>.</p>
<p>This should render the help page if you navigate to
<code class="language-plaintext highlighter-rouge">http://localhost:8080/help</code></p>
<figure>
<img src="img/basic-help-page.png" alt="basic help page" />
</figure>
<p>If you are having trouble viewing the page you just created, you can try to clear the view cache.</p>
<p><code class="language-plaintext highlighter-rouge">> php artisan view:clear</code></p>
<p>The last step, we want to modify the navigate menu blade, to add the link to the menu to get to our new page.</p>
<h2 id="add-link-to-menu">Add link to menu</h2>
<p>Add the link to the Navigation menu blade
Navigate to this file <code class="language-plaintext highlighter-rouge">/resources/views/navigation-menu.blade.php</code></p>
<p>You should see this section of code on lines 13-18.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><!-- Navigation Links -->
<div class="hidden space-x-8 sm:-my-px sm:ml-10 sm:flex">
<x-jet-nav-link href="" :active="request()->routeIs('dashboard')">
</x-jet-nav-link>
</div>
</code></pre></div></div>
<p>We want to simply add a block of code to this navigation links part.
As I want adding a help page, I added the following code block for a help page.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <div class="hidden space-x-8 sm:-my-px sm:ml-10 sm:flex">
<x-jet-nav-link href="" :active="request()->routeIs('help')">
</x-jet-nav-link>
</div>
</code></pre></div></div>
<p>The full navigation link block should look like this</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <!-- Navigation Links -->
<div class="hidden space-x-8 sm:-my-px sm:ml-10 sm:flex">
<x-jet-nav-link href="" :active="request()->routeIs('dashboard')">
</x-jet-nav-link>
</div>
<div class="hidden space-x-8 sm:-my-px sm:ml-10 sm:flex">
<x-jet-nav-link href="" :active="request()->routeIs('help')">
</x-jet-nav-link>
</div>
<!-- \Navigation Links -->
</code></pre></div></div>
<p>Now, you should see the help link on the navigation menu.</p>
<figure>
<img src="img/help-nav-menu.png" alt="Help nav menu" />
</figure>
<p>All done.
You should now see your new help page with guide and support components.</p>
<figure>
<img src="img/LaravelJetstreamHelpPage.gif" alt="Laravel Jetstream Help Page" />
</figure>
<h1 id="further-laravel-resources">Further Laravel Resources</h1>
<p>Some other resources which might be of use once you have started with Laravel can be found in the
list below:</p>
<ul>
<li><a href="/category/guides/script-increment-laravel-version/">Script to Programmatically Increment Laravel Application Version</a></li>
<li><a href="/category/dev/quick-and-easy-way-to-test-sending-email-on-laravel/">Quick and easy way to test sending email on Laravel</a></li>
<li><a href="/category/dev/visually-distinguish-build-environments/">How to visually distinguish build environments in Laravel</a></li>
<li><a href="/category/guides/local-development-environments/#laravel">Laravel Development Environments</a></li>
</ul>Ayesh AlshukriDevelopment Environment Using Docker for Laravel Jetstream Applications2021-06-12T00:00:00+01:002021-06-12T00:00:00+01:00https://ayeshalshukri.co.uk/category/guides/development-environment-docker-lemp-laravel-jetstream<figure>
<img src="LaravelJetstream.webp" alt="main page image" />
</figure>
<p>This article provides details on how to setup development environment
using Docker implementing a LEMP server used for developing Laravel Jetstream applications.</p>
<blockquote>
<p><a href="#quick-start-guide">Skip to the quick start guide with code download</a></p>
</blockquote>
<p>This guide is a step by step to get everything up and running as a basis to create an application.</p>
<p>I will be using Laravel Jetstream with the Livewire + Blade stack (which utilises Tailwind CSS)</p>
<p>Features out of the box include:</p>
<ul>
<li>Authentication</li>
<li>Registration</li>
<li>Profile Management</li>
<li>Password Update</li>
<li>Password Confirmation</li>
<li>Two Factor Authentication</li>
<li>Browser Sessions</li>
<li>API</li>
<li>Teams</li>
</ul>
<!--
In particular, I used this approach for recent projects in the list below which you can find on [my projects page](/projects/).
-->
<p>The contents of this article will cover stages:</p>
<ul>
<li><a href="#what-is-laravel-jetstream">What is Laravel Jetstream</a></li>
<li><a href="#tech-stack">Tech Stack</a></li>
<li><a href="#setup-development-environment">Setup Development Environment</a></li>
<li><a href="#installing-laravel">Installing Laravel</a></li>
<li><a href="#application-screenshots">Application Screenshots</a></li>
<li><a href="#quick-start-guide">Quick Start Guide</a></li>
</ul>
<h1 id="what-is-laravel">What is Laravel?</h1>
<p>Laravel is a framework for the PHP programming language.
You can read more in my
<a href="/category/dev/php-laravel-framework/">guide to getting started with PHP Laravel Framework</a></p>
<h1 id="what-is-laravel-jetstream">What is Laravel Jetstream?</h1>
<p>What is Laravel Jetstream?
Laravel Jetstream is an open-source package developed by the Laravel team.</p>
<p>Is laravel Jetstream free?
Yes, the Jetstream package is free, as is the Laravel framework it is developed on to of.
The laravel Jetstream package contains the basic authentication scaffolding and more.</p>
<p>I will be using Laravel Jetstream with the Livewire + Blade stack (which utilises Tailwind CSS)</p>
<h4 id="what-are-teams-in-laravel-jetstream">What are teams in laravel Jetstream?</h4>
<p>In Laravel jetstream the ‘teams’ functionality is built on top of the basic user management.
Teams add functionality to this basic structure.
It adds users roles, permissions and allows users to be associated with a ‘team’.</p>
<h6 id="user-profile-management">User Profile Management</h6>
<p>User management includes:</p>
<ul>
<li>Display and update profile picture.</li>
<li>Contains profile name, along with email and password.</li>
<li>Records Browser Sessions and allow session control (like log out other devices).</li>
<li>Option for users to delete account.</li>
<li>Two Factor Authentication</li>
</ul>
<h6 id="team-management">Team Management</h6>
<p>Teams management includes:</p>
<ul>
<li>Create a Team</li>
<li>Add and update users to a team</li>
<li>Manage Team, update team display name.</li>
<li>User can Switch Team between multiple teams they belong to.</li>
<li>Manage the role and permission of the members of a team.</li>
</ul>
<h1 id="tech-stack">Tech Stack</h1>
<p>I am going to be using a few different technologies for this particular setup.</p>
<p>For the server, fairly standard:</p>
<ul>
<li>Ubuntu Linux</li>
<li>MySQL</li>
</ul>
<p>This would technically count as a LEMP stack.
[LEMP] = Ubuntu + Nginx + MySQL + PHP</p>
<p>For the application, I am going to be using:</p>
<ul>
<li>Laravel</li>
<li>Jetstream</li>
<li>livewire</li>
<li>tailwind css</li>
<li>fortify</li>
</ul>
<p>The next section will cover the setup of my local development environment.</p>
<h1 id="setup-development-environment">Setup Development Environment</h1>
<p>The first step is to setup our local development environment (DevEnv).
I often use docker to achieve this.</p>
<p>The development environment used in this project is:</p>
<ul>
<li>L - Linux</li>
<li>E - Nginx</li>
<li>M - MySQL</li>
<li>P - PHP</li>
</ul>
<p>This docker container along with basic Laravel install is available
from this repository.</p>
<p><a id="github_DevEnv-Docker-LEMP-Laravel_Jetstream" target="_blank" rel="noopener" href="https://github.com/aalshukri/DevEnv-Docker-LEMP-Laravel_Jetstream">
https://github.com/aalshukri/DevEnv-Docker-LEMP-Laravel_Jetstream</a></p>
<p>You can read more about the
<a href="">various other development environments</a>
I have used in the past.</p>
<h1 id="installing-laravel">Installing Laravel</h1>
<p>I will be using php framework Laravel to develop this application.</p>
<p>I started up my DevEnv container, and then connected to the Linux terminal running inside the docker container. (see the connect.sh script)</p>
<p>This would then allow me to install a fresh version of the Laravel (https://laravel.com/) using composer via the command line.</p>
<p>Below is the set of instruction I used to achieve this.</p>
<p>Installation Via Composer</p>
<p><code class="language-plaintext highlighter-rouge">> composer create-project laravel/laravel <project-name></code>
<code class="language-plaintext highlighter-rouge">> composer create-project ---prefer-dist laravel/laravel </code></p>
<p>You will get output similar to the following:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Creating a "laravel/laravel" project at "./<project-name>"
Installing laravel/laravel (v8.5.17)
- Installing laravel/laravel (v8.5.17): Downloading (100%)
Created project in /<project-name>
...
Loading composer repositories with package information
...
Updating dependencies (including require-dev)
Package operations: 104 installs, 0 updates, 0 removals
...
Writing lock file
Generating optimized autoload files
...
> @php artisan key:generate --ansi
Application key set successfully.
</code></pre></div></div>
<p>After the installation was completed,
you can check everything has worked so far by navigating to your project folder, and running the following command.</p>
<p><code class="language-plaintext highlighter-rouge">> php artisan --version</code></p>
<p>You should get output something like below, showing you the version
of the Laravel framework used.</p>
<p><code class="language-plaintext highlighter-rouge">Laravel Framework 8.41.0</code></p>
<p>If you get something like the following error, then it is likely you have some problems with the path of your composer installation.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>php artisan --version
PHP Warning: require(vendor/autoload.php): failed to open stream: No such file or directory in artisan on line 18
PHP Fatal error: require(): Failed opening required 'vendor/autoload.php' (include_path='.:/usr/share/php') in artisan on line 18
</code></pre></div></div>
<p>You can try and run the composer install command and then update composer paths to solve this problem.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>> cd <project-name>
> composer install
> composer dump-autoload
</code></pre></div></div>
<h2 id="setup-laravel">Setup Laravel</h2>
<p>Setup you .env file.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>composer dump-autoload
php artisan cache:clear
php artisan config:clear
php artisan view:clear
php artisan optimize:clear
</code></pre></div></div>
<h1 id="install-livewire--jetstream--fortify">Install Livewire + Jetstream + Fortify</h1>
<h4 id="livewire">Livewire</h4>
<p>Install livewire</p>
<p><code class="language-plaintext highlighter-rouge">> composer require livewire/livewire</code></p>
<p><code class="language-plaintext highlighter-rouge">> php artisan livewire:publish --config</code></p>
<h4 id="jetstream">Jetstream</h4>
<p>To install Jetstream library,
first install via composer:</p>
<p><code class="language-plaintext highlighter-rouge">> composer require laravel/jetstream</code></p>
<p>If you get the following error, then you might need to change a few permissions.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>file_put_contents(./composer.json): failed to open stream:
Operation not permitted
</code></pre></div></div>
<p>To change composer permissions, issue the command</p>
<p><code class="language-plaintext highlighter-rouge">sudo chown -R $USER ~/.composer/</code></p>
<p>chown -R root ~/.composer/</p>
<p>To configure jetstream, with the option of teams.</p>
<p><code class="language-plaintext highlighter-rouge">> php artisan jetstream:install livewire --teams</code></p>
<p>Then we need to do a application build.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>npm install
npm run dev
</code></pre></div></div>
<p>If you are having problems with <code class="language-plaintext highlighter-rouge">npm run dev</code></p>
<p>update to new version of nodejs.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cd ~
curl -sL https://deb.nodesource.com/setup_14.x -o nodesource_setup.sh
bash nodesource_setup.sh
apt install nodejs
</code></pre></div></div>
<p>Then configure the database.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>php artisan migrate:fresh
</code></pre></div></div>
<p>Then setup the blade views and routes.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>php artisan vendor:publish --tag=jetstream-views
php artisan vendor:publish --tag=jetstream-routes
</code></pre></div></div>
<p>php artisan route:clear</p>
<h4 id="fortify">Fortify</h4>
<p>composer require laravel/fortify</p>
<p>php artisan vendor:publish –provider=”Laravel\Fortify\FortifyServiceProvider”</p>
<h4 id="possible-errors-during-installation">Possible Errors during installation</h4>
<p>laravel jetstream 404 current-team</p>
<p>laravel jetstream 404 create new team</p>
<p>laravel jetstream 404 update team</p>
<p>laravel jetstream cannot create new team</p>
<p>laravel jetstream cannot update current team name</p>
<p>laravel jetstream cannot update profile picture</p>
<p>I did get some of these problems.
To resolve, I finally found the problem to be the livewire js not loading.
you might need to check the asset url <code class="language-plaintext highlighter-rouge">ASSET_URL</code>
in livewire config file
<code class="language-plaintext highlighter-rouge">/www/config/livewire.php</code>.
This solved my problem of the JS loading.</p>
<h2 id="folder-structure">Folder Structure</h2>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>.
├── README.md
├── docker
│ ├── build.sh
│ ├── config.sh
│ ├── connect.sh
│ ├── docker-compose.yml
│ ├── phpdocker
│ ├── setup
│ ├── start.sh
│ └── stop.sh
└── www
├── app
├── artisan
├── bootstrap
├── composer.json
├── composer.lock
├── config
├── database
├── node_modules
├── package-lock.json
├── package.json
├── phpunit.xml
├── public
├── resources
├── routes
├── server.php
├── storage
├── tailwind.config.js
├── tests
├── vendor
└── webpack.mix.js
</code></pre></div></div>
<h1 id="application-screenshots">Application Screenshots</h1>
<p>Once you have completed the steps above, the application that you have should
look like the screenshots below.</p>
<p>laravel jetstream welcomepage</p>
<figure>
<img src="img/laravel-jetstream-welcomepage.png" alt="" />
</figure>
<p>laravel jetstream registerpage</p>
<figure>
<img src="img/laravel-jetstream-registerpage.png" alt="" />
</figure>
<p>laravel-jetstream-tospage.png</p>
<figure>
<img src="img/laravel-jetstream-tospage.png" alt="" />
</figure>
<p>laravel-jetstream-pppage.png</p>
<figure>
<img src="img/laravel-jetstream-pppage.png" alt="" />
</figure>
<p>laravel-jetstream-loginpage.png</p>
<figure>
<img src="img/laravel-jetstream-loginpage.png" alt="" />
</figure>
<h4 id="jetstream-login-process">jetstream login process</h4>
<figure>
<img src="img/LaravelJetstreamLogin/laravel-jetstream-login.gif" alt="" />
</figure>
<h4 id="user-profile">user profile</h4>
<figure>
<img src="img/LaravelJetStreamUserProfile/userprofile.gif" alt="" />
</figure>
<h4 id="teams">teams</h4>
<figure>
<img src="img/LaravelJetStreamTeams/team.gif" alt="" />
</figure>
<h1 id="quick-start-guide">Quick Start Guide</h1>
<p><a id="quick-start-guide" target="_blank" rel="noopener" href="https://github.com/aalshukri/DevEnv-Docker-LEMP-Laravel_Jetstream">
https://github.com/aalshukri/DevEnv-Docker-LEMP-Laravel_Jetstream</a></p>
<p>To get this docker container up and running simply build and start using the following commands
from within the <code class="language-plaintext highlighter-rouge">docker</code> directory.</p>
<p><code class="language-plaintext highlighter-rouge">> cd docker</code></p>
<p><code class="language-plaintext highlighter-rouge">> ./build.sh</code></p>
<p><code class="language-plaintext highlighter-rouge">> ./start.sh</code></p>
<p><code class="language-plaintext highlighter-rouge">> ./config.sh</code></p>
<p>You can access the application at
<code class="language-plaintext highlighter-rouge">https://127.0.0.1:8080/</code> or <code class="language-plaintext highlighter-rouge">https://localhost:8080/</code></p>
<p>Note; the <code class="language-plaintext highlighter-rouge">build.sh</code> and <code class="language-plaintext highlighter-rouge">config.sh</code> commands are only needs the first time, but won’t harm to run each time.
Once finished call the stop command to bring down docker container and application.</p>
<p><code class="language-plaintext highlighter-rouge">> ./stop.sh</code></p>
<p>While the docker container is running, ie after running <code class="language-plaintext highlighter-rouge">./start</code> you can
connect to the web server to run commands as if you would any other server.
Use the following command to connect</p>
<p><code class="language-plaintext highlighter-rouge">> ./connect.sh</code></p>
<p>The following section provides further details on the docker methods used within each script.</p>
<h2 id="application-access">Application Access</h2>
<ul>
<li>Main Application: <code class="language-plaintext highlighter-rouge">http://localhost:8080/</code></li>
<li>PhpMyAdmin: <code class="language-plaintext highlighter-rouge">http://localhost:8181/</code></li>
<li>Email (SMPT catcher) <code class="language-plaintext highlighter-rouge">http://localhost:3000/</code></li>
</ul>
<h1 id="database">Database</h1>
<p>MySQL and PHPMyAdmin</p>
<h4 id="phpmyadmin">PhpMyAdmin</h4>
<p>You can access the database using phpmyadmin within this project.
Navigate to this url <code class="language-plaintext highlighter-rouge">http://127.0.0.1:8181/index.php</code></p>
<p>Enter the following details</p>
<ul>
<li>Server: webapp-mysql</li>
<li>Username: webapp_user</li>
<li>Password: password</li>
</ul>
<h2 id="acknowledgements">Acknowledgements</h2>
<p>This project is based on
<a href="https://github.com/aalshukri/DevEnv-Docker-LEMP-Laravel">https://github.com/aalshukri/DevEnv-Docker-LEMP-Laravel</a>.</p>
<p>Main differences between DevEnv-Docker-LEMP-Laravel and DevEnv-Docker-LEMP-Drupal
are the following:</p>
<ul>
<li>Ubuntu 20.04 from 18.04</li>
<li>PHP 7.4 (php7.4-fpm.sock) from 7.2</li>
</ul>Ayesh AlshukriGuide to Getting Started with PHP Laravel Framework2021-06-12T00:00:00+01:002021-06-12T00:00:00+01:00https://ayeshalshukri.co.uk/category/dev/php-laravel-framework<figure>
<img src="laravel.png" alt="main page image" />
</figure>
<h1 id="what-is-laravel">What is Laravel?</h1>
<p>I often get asked many questions like,
What is PHP laravel used for?
What is difference between PHP and laravel?
Is laravel pure PHP?
Is laravel frontend or backend?
Is laravel easier than PHP?
Can I learn laravel without PHP?</p>
<p>I have put this article together to answer these questions.</p>
<blockquote>
<p><a href="#getting-started">Skip to the getting started part of the guide contained links to code</a></p>
</blockquote>
<p>Laravel is a framework.
PHP is programming language.
A framework is a way of structuring and organisation your code.
In this case, Laravel is a way of structuring and organisation your PHP code.</p>
<p>As you can see, the difference between PHP and laravel,
is that PHP is a programming, laravel is a way of ‘writing’ you PHP code.
As a result, you cannot learn laravel without PHP.</p>
<p>Often, frameworks follow a particular software design pattern.
In this case, Laravel follows a model–view–controller (usually known as MVC) design pattern.</p>
<ul>
<li>Model - Data, MySQL database</li>
<li>View - Interface user interacts with. Buttons, tables etc.</li>
<li>Controller - Logic.</li>
</ul>
<p>There are many frameworks for the php programming language, Laravel is popular and well supported.</p>
<h1 id="what-is-php-laravel-used-for">What is PHP laravel used for?</h1>
<p>Laravel is a very powerful and flexible framework.
It can be used for both front end, and backend applications.</p>
<p>Laravel does offer some basic frontend functionality out of the box.
The main feature being user management.
The basic user management allows users to register, login, email reset password etc.</p>
<p>The user management is basic.
Often times, you might need more than this.
Functionality like user roles, permissions and teams.
This can be offered by
<a href="/category/guides/development-environment-docker-lemp-laravel-jetstream/">laravel Jetstream</a>.</p>
<h1 id="is-laravel-easy-to-learn">Is Laravel easy to learn?</h1>
<p>My Step by step guide to getting started (with code).</p>
<p>I have put together a comprehensive step by step guide to get up and running with links to the
basic code needed for building a new application in Laravel.
This starter code has functionality for user login, teams, profile pictures, API tokens,
two-factor authentication (2FA) etc (as shown in the image below).</p>
<h1 id="getting-started">Getting Started</h1>
<p>This getting started section contains links to the main resources
I have on my site to help you getting started with Laravel.</p>
<h5 id="basic-laravel-application">Basic Laravel Application</h5>
<p><a href="/category/guides/development-environment-docker-lemp-laravel/">
This guide contains a docker development environment
</a> and a basic install of Laravel (DevEnv-Docker-LEMP-Laravel).</p>
<h5 id="laravel-jetstream">Laravel Jetstream</h5>
<p>This guide contains
<a href="/category/guides/development-environment-docker-lemp-laravel-jetstream/">
docker development environment and a basic
installation of Laravel Jetstream</a> which provides user login, teams, profile pictures, API tokens,
two-factor authentication (2FA) out of the box.</p>
<p>DevEnv-Docker-LEMP-Laravel_Jetstream
Jetstream
using Livewire
fortify
tailwind css</p>
<h1 id="further-resources">Further Resources</h1>
<p>Some other resources which might be of use once you have started with Laravel can be found in the
list below:</p>
<ul>
<li><a href="/category/guides/script-increment-laravel-version/">Script to Programmatically Increment Laravel Application Version</a></li>
<li><a href="/category/dev/quick-and-easy-way-to-test-sending-email-on-laravel/">Quick and easy way to test sending email on Laravel</a></li>
<li><a href="/category/dev/visually-distinguish-build-environments/">How to visually distinguish build environments in Laravel</a></li>
<li><a href="/category/guides/local-development-environments/#laravel">Laravel Development Environments</a></li>
</ul>Ayesh AlshukriREDCap ‘Hello World’ external module development guide2021-05-10T00:00:00+01:002021-05-10T00:00:00+01:00https://ayeshalshukri.co.uk/category/guides/redcap-external-module-development-guide-hello-world<!--
<figure>
<img src="development-environment-docker-lemp-laravel.gif" alt="development environment docker lemp laravel">
</figure>
-->
<p>This development guide covers how to create an external module in REDCap.
The external module developed in this guide is a simple (classic) ‘hello world’ example.</p>
<blockquote>
<p><a href="#basic-hello-world-external-module">Skip to the hello world external module example (with code!)</a></p>
</blockquote>
<p>How to get started creating an external module in REDCap?</p>
<p>Setup a development environment on your local machine.
This should replicate your production environment as closely as possible.
I do this using docker, here are some links which may help you get up and running
on my <a href="/category/guides/local-development-environments/">development environments page</a>.</p>
<p>You could set up one of these development environments,
and then copy over the version of REDCap you are going to be using.
In my case, I am going to be working with
a few different versions to test compatibility:</p>
<ul>
<li>v8.8.2</li>
<li>v9.1.8</li>
<li>v10.6.17</li>
</ul>
<p>Once you have your local dev env setup,
the first step is to download the official REDCap external module template.
You can download the files here
<a href="https://github.com/vanderbilt-redcap/external-module-templater">
https://github.com/vanderbilt-redcap/external-module-templater</a></p>
<p>Then, navigate to your external modules folder
In my case, this was
here for v9.1.8</p>
<p><code class="language-plaintext highlighter-rouge">www/redcap/redcap_v9.1.8/ExternalModules/example_modules</code></p>
<p>and similarly, here for v10.6.17</p>
<p><code class="language-plaintext highlighter-rouge">www/redcap/redcap_v10.6.17/ExternalModules/example_modules</code></p>
<p>create a new folder, called</p>
<p><code class="language-plaintext highlighter-rouge">companyname_ExternalModuleName_v0.1</code></p>
<p>Then copy the content of the <code class="language-plaintext highlighter-rouge">external-module-templater-master</code> folder.
You should end up with a folder structure like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>├── companyName_ExternalModuleName_v0.1
│ ├── LICENSE
│ ├── README.md
│ ├── Templater.php
│ ├── composer.json
│ ├── composer.lock
│ ├── config.json
│ ├── generate_template.php
│ ├── js
│ │ └── functions.js
│ └── templates
│ ├── README.twig
│ ├── class.twig
│ ├── config.twig
│ └── newModule.twig
</code></pre></div></div>
<h1 id="how-to-install-external-modules-in-redcap">How to install External Modules in REDCap?</h1>
<p>To check if the previous steps actually worked,
we need to install the external module in redcap.
This will mean it can be seen and used by REDCap.
In order to do this,
navigate to control center,
then from the left hand menu, select external modules, and enable a module.</p>
<figure>
<img src="img/RedcapEnableExternalModule.gif" alt="Redcap Enable External Module" />
</figure>
<p>You should see your new external module in this list,
as shown below.</p>
<figure>
<img src="img/REDCap-EnableModule.png" alt="REDCap Enable Module" />
</figure>
<p>You have now completed the basic steps to setup a REDCap external module.
The next section of this article has code for a basic hello world external module.</p>
<h1 id="basic-hello-world-external-module">Basic Hello World External Module</h1>
<p>How do we setup a very basic hello world external module?</p>
<p>Note; this code was developed using REDCap Version 10.6.17.
All code is available from my
<a href="https://github.com/aalshukri/REDCap_ExternalModule_HelloWorld">
github repository github.com/aalshukri/REDCap_ExternalModule_HelloWorld”</a>.</p>
<p>Create a directory with the format
<code class="language-plaintext highlighter-rouge"><company>_<module name>_<version number></code>
An example, would be
<code class="language-plaintext highlighter-rouge">company_HelloWorldModule_v0.1</code></p>
<p>Download the files from my repository
<a href="https://github.com/aalshukri/REDCap_ExternalModule_HelloWorld">https://github.com/aalshukri/REDCap_ExternalModule_HelloWorld</a>.</p>
<p>You want to look at two files that are important to the running of this hello world example:</p>
<ul>
<li>config.json</li>
<li>HelloWorldExternalModule.php</li>
</ul>
<p>The first file, is the <code class="language-plaintext highlighter-rouge">config.json</code>.</p>
<p>This file has been updated with information on the external module, like description and author etc.
But the really important part of the <code class="language-plaintext highlighter-rouge">config.json</code> is the <code class="language-plaintext highlighter-rouge">permissions</code> part.
Note, how the module is requesting permission for <code class="language-plaintext highlighter-rouge">redcap_data_entry_form</code>.
This is also reflected in the function call in HelloWorldExternalModule.php.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> "permissions": [
"redcap_data_entry_form"
],
</code></pre></div></div>
<p>Here is a sample of the HelloWorldExternalModule.php file.
You will notice, the main function is named <code class="language-plaintext highlighter-rouge">redcap_data_entry_form</code>.
The line <code class="language-plaintext highlighter-rouge">print '<div class="yellow">Hello World.</div>';</code> simply acts as our hello world print statement,
which is output as a line on the bottom of all input forms within the REDCap project.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> function redcap_data_entry_form(...)
{
print '<div class="yellow">Hello World.</div>';
}
</code></pre></div></div>
<p>Now we have taken a look at the important files,
we can look at enabling this module so we can see it working on REDCap.</p>
<p>Navigate to the control centre home in your REDCap instance and enable your ‘Hello World’ external module.</p>
<figure>
<img src="img/REDCapExternalModulesHelloWorld_Enable.gif" alt="REDCap External Modules Hello World Enable" />
</figure>
<p>Once enabled, you should see your module listed as available on the system.</p>
<figure>
<img src="img/REDCapExternalModuleHelloWorld_Available.png" alt="REDCap External Module Hello World" />
</figure>
<p>Once enabled on the REDCap system, we then need to enabled our hello world external module on a
specific project.
Navigate to the ‘External Module’ section.</p>
<figure>
<img src="img/REDCapProjectExternalModulesMenu.png" alt="REDCap Project External Modules Menu" />
</figure>
<figure>
<img src="img/REDCapProjectExternalModulesPage.png" alt="REDCap Project External Modules Page" />
</figure>
<p>From this page, you can now enable this module for your project.</p>
<figure>
<img src="img/REDCapExternalModulesHelloWorld_ProjectEnable.gif" alt="REDCap External Modules Hello World Project Enable" />
</figure>
<p>Once this is enabled, then you can navigate to one of your projects CRFs,
you should see the ‘hello world’ text displayed at the bottom of the data entry form, as shown below.</p>
<figure>
<img src="img/REDCapExternalModuleHelloWorld_Running.png" alt="REDCap External Module Hello World Running" />
</figure>
<p>If you found this
redcap external module development guide useful,
and would like to see some of the other REDCap external modules I have built,
or need advice, please reach out and <a href="/contact">contact me</a>.</p>Ayesh AlshukriLocal Development Environments2021-04-30T00:00:00+01:002021-04-30T00:00:00+01:00https://ayeshalshukri.co.uk/category/guides/local-development-environments<!--
<figure>
<img src="" alt="DevOps Laravel Version Script">
</figure>
-->
<p>This article lists all the development environments I have created
for local application dev using docker.
The following section are organised by programming language,
and then framework or other related characteristic.</p>
<ul>
<li><a href="#php">PHP</a>
<ul>
<li><a href="#laravel">Laravel</a></li>
<li><a href="#drupal">Drupal</a></li>
<li><a href="#php-with-xdebug-enabled">Xbebug</a></li>
</ul>
</li>
<li><a href="#python">Python</a></li>
</ul>
<h2 id="php">PHP</h2>
<h4 id="laravel">Laravel</h4>
<p>The following development environments have been created specifically for Laravel</p>
<p>Development Environment using Docker implementing a LEMP server used for hosting Laravel applications.
[LEMP] = Ubuntu 18.04 Base Image + Nginx + MySQL + PHP 7.2</p>
<p>DevEnv-Docker-LEMP-Laravel
<a href="https://github.com/aalshukri/DevEnv-Docker-LEMP-Laravel">https://github.com/aalshukri/DevEnv-Docker-LEMP-Laravel</a></p>
<p>Read more here in my guide for
<a href="/category/guides/development-environment-docker-lemp-laravel/">development environment for Laravel</a>.</p>
<h4 id="drupal">Drupal</h4>
<p>Development Environment using Docker implementing a LEMP server used for hosting Drupal applications.
[LEMP] = Ubuntu 20.04 Base Image + Nginx + MySQL + PHP 7.4</p>
<p>DevEnv-Docker-LEMP-Drupal
<a href="https://github.com/aalshukri/DevEnv-Docker-LEMP-Drupal">https://github.com/aalshukri/DevEnv-Docker-LEMP-Drupal</a></p>
<h4 id="php-with-xdebug-enabled">PHP with Xdebug Enabled</h4>
<p>This is a development environment for PHP projects using PhpStorm and Docker with xdebug enabled.
(note, the title suggests Laravel, but this is DevEnv is framework independent).</p>
<p>Read more here in my guide for
<a href="/category/guides/local-development-environment-php-laravel-applications/">development environment for php with xdebug</a>.</p>
<h2 id="python">Python</h2>
<p>Development Environment using Docker implementing python3 used for developing python applications.</p>
<p>ipython
requests
numpy
pandas
scikit-learn
scipy
boto3</p>
<p>DevEnv-Docker-Python3
<a href="https://github.com/aalshukri/DevEnv-Docker-Python3">https://github.com/aalshukri/DevEnv-Docker-Python3</a></p>
<hr />
<hr />
<p>You can read more about how I have used the
<a id="articleend" href="/tags/#docker">docker</a>,
<a id="articleend" href="/tags/#python">python</a>,
<a id="articleend" href="/tags/#php">php</a>
(<a id="articleend" href="/tags/#laravel">laravel</a>, <a id="articleend" href="/tags/#drupal">drupal</a>)
technolgogies
and <a id="articleend" href="/tags/#devops">devops</a> methodologys in my
other projects.</p>
<p>If you have any questions or comments you can <a id="ContactMe" href="/contact/">contact me</a>.</p>Ayesh AlshukriDocker introduction for non technical audience2021-04-26T00:00:00+01:002021-04-26T00:00:00+01:00https://ayeshalshukri.co.uk/category/guides/docker-intro-non-tech-audience<figure>
<img src="DockerDeployment.png" alt="Docker" />
</figure>
<p>This article provides an introduction to Docker for a non-technical audience.</p>
<blockquote>
<p>Why would non-technical people care about docker?</p>
</blockquote>
<p>Within my organisation, like many others,
we have mixed technical and non technical team members.
Within the tech team, we are often using jargon and tech terms.
I think it is important to educate and make sure, all staff,
no matter tech background, can understand to a basic level, what we are talking about.</p>
<h1 id="docker-as-devenv">Docker as DevEnv</h1>
<p>Docker is actually a large and quite complex application and eco system.
There are many advanced features.
For this particular article, I want to narrow down the focus to a particular use case of docker,
which is to help us setup our computers and laptops ready for application testing and development.
This particular use case is often referred to as a
development environment (<a href="/tags/#development-environment">DevEnv</a>).</p>
<h5 id="what-do-i-mean-by-that">What do I mean by that?</h5>
<p>Essentially, what I can going to talk about is how we can use docker to create a
virtual machine on your laptop that acts like a server.
Docker can help us achieve this in a consistent and efficient manner.</p>
<p>This is a simplified scenario, but typically,
if we want to develop an application, we need a server, with two
main components, the web service part, and the database part.</p>
<h5 id="docker-devenv-components">Docker DevEnv Components</h5>
<p>I am going to use a
<a href="https://github.com/aalshukri/DevEnv-Docker-LEMP-Laravel">php laravel</a>
example that you can read more about if interested.
In this explanation, I am going to concentrate on two main components:</p>
<ul>
<li>Web service part is so we can www.application.com and get a result.</li>
<li>Database part is so we can store/retrieve information.</li>
</ul>
<p>I am going to explain with these components using two files.
There is a lot more happening here, but for simplicity, I am just going to concentrate on two files.
A Dockerfile and a docker-compose.yml file.
These are both in yaml format, so are easily readable.</p>
<p><a href="https://github.com/aalshukri/DevEnv-Docker-LEMP-Laravel/blob/main/docker/lemp-base/Dockerfile">Dockerfile sample</a></p>
<p><a href="https://github.com/aalshukri/DevEnv-Docker-LEMP-Laravel/blob/main/docker/docker-compose.yml">docker-compose.yml sample</a></p>
<h1 id="dockerfile">Dockerfile</h1>
<p>As you can see from the Dockerfile,
this describes the basic server setup.
This is the virtual server that will be setup on your laptop when starting docker with these files.</p>
<p>Three main parts I want to draw your attention to.
Lines 1 and 2.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>#Ubuntu 18.04 Base Image - Nginx + MySQL + PHP 7.2 [LEMP]
FROM ubuntu:18.04
...
</code></pre></div></div>
<p>This described the type of server operating system.
This could be CentOS etc, but in this case, it is Ubuntu.</p>
<p>Lines 42 to 50 show certain php libraries being installed.
In this case, they are installed as per the latest version for this operating system, Ubuntu 20.04.
But, depending on OS, we can specific the version.
This would be something we are going to look at in order to replicate our production servers and closely as possible.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>...
php-fpm \
php-cli \
php-common \
php-zip \
php-mbstring \
php-bcmath \
php-image-text \
php-gd \
php-mysql \
...
</code></pre></div></div>
<p>Lines 53 to 55 show the installation of database,
in this case, mysql.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>...
mysql-server \
mysql-client \
mysql-common \
...
</code></pre></div></div>
<p>Here is a look at the entire dockerfile in this example.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>#Ubuntu 18.04 Base Image - Nginx + MySQL + PHP 7.2 [LEMP]
FROM ubuntu:18.04
#Label
LABEL version="1.0"
LABEL name="ubuntu-base+nginx+mysql+php"
#Set TZ
ENV TZ=Europe/London
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
#apt update & install
RUN apt-get update && apt-get install -y \
git \
curl \
libpng-dev \
libonig-dev \
libxml2-dev \
libzip-dev \
zip \
unzip \
graphviz \
npm \
libxpm4 \
libxrender1 \
libgtk2.0-0 \
libnss3 \
libgconf-2-4 \
chromium-browser \
xvfb \
gtk2-engines-pixbuf \
xfonts-cyrillic \
xfonts-100dpi \
xfonts-75dpi \
xfonts-base \
xfonts-scalable \
imagemagick x11-apps \
libwebp-dev \
libfreetype6-dev \
libjpeg62-dev \
libpng-dev libxpm-dev \
php-fpm \
php-cli \
php-common \
php-zip \
php-mbstring \
php-bcmath \
php-image-text \
php-gd \
php-mysql \
composer \
nginx \
mysql-server \
mysql-client \
mysql-common \
php-xml \
php-curl
</code></pre></div></div>
<h1 id="docker-compose">docker-compose</h1>
<p>A few keys points from the docker-compose file.
If you look at the first block, this is essentially starting up the server,
using the image as you have just see in previous section dockerfile
(it is actually a bit more complicated than that, but, for simplicity).</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>...
# LEMP Server from image
app:
image: lemp-dev:v2
container_name: webapp
env_file:
- .env
working_dir: /var/www/html/
ports:
- 80:80
- 443:443
- 3306:3306
volumes:
- ../www:/var/www/html/
networks:
- lempnet
tty: true
...
</code></pre></div></div>
<p>There are some settings that you don’t have to understand for the purposes of our use case in the section.</p>
<p>The second part is starting phpmyadmin from the official image on the docker hub web repository.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>...
# DB access via PHP my admin
phpmyadmin:
image: phpmyadmin
container_name: phpmyadmin
environment:
- PMA_ARBITRARY=1
restart: always
ports:
- 8181:80
networks:
- lempnet
...
</code></pre></div></div>
<p>Here is a look at the complete docker-compose file</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>version: "3.3"
networks:
# docker internal network
lempnet:
driver: bridge
services:
# LEMP Server from image
app:
image: lemp-dev:v2
container_name: webapp
env_file:
- .env
working_dir: /var/www/html/
ports:
- 80:80
- 443:443
- 3306:3306
volumes:
- ../www:/var/www/html/
networks:
- lempnet
tty: true
# DB access via PHP my admin
phpmyadmin:
image: phpmyadmin
container_name: phpmyadmin
environment:
- PMA_ARBITRARY=1
restart: always
ports:
- 8181:80
networks:
- lempnet
</code></pre></div></div>
<h1 id="docker-desktop">Docker desktop</h1>
<p>Here is a look what the docker desktop application looks like on MacOS.</p>
<figure>
<img src="docker-desktop-mac.png" alt="Docker desktop mac" />
</figure>
<p>You can read more about how I have used <a href="/tags/#docker">docker in my other projects</a>.</p>Ayesh AlshukriGuide to using confluence: organisation and structure2021-04-08T09:00:00+01:002021-04-08T09:00:00+01:00https://ayeshalshukri.co.uk/category/guides/guide-confluence<!--
<figure>
<img src="" alt="Python for data science">
</figure>
-->
<p>We have decided to use Atlassian’s Confluence system for our
internal wiki / knowledge base (KB).
This system is often referred to in our internal documentation as:</p>
<ul>
<li>IT Wiki</li>
<li>IT Knowledge Base (KB)</li>
<li>IT Confluence</li>
</ul>
<p>If you see any of these references in an SOP or other documentation,
then rest assured, it is in reference to our Atlassian Confluence system.</p>
<h1 id="what-is-confluence">What is Confluence?</h1>
<p>Atlassian’s Confluence software is “enterprise knowledge management system”[1]</p>
<p>Essentially this is a set of web pages containing information that is developed,
edited collaboratively and ultimately managed by its own audience.
(see definition of wiki [2]).</p>
<p>Note; Atlassian is a commercial company.
One of many product offerings they have is Confluence.</p>
<p>Atlassian Revenue $1.61 billion in July 2020 [1]. as a result,
there are many aspects that are paid for.
Some simple functionality that you would expect to be standard,
is actually a paid for option. As you will see in the following demo.
(Hint: Archive a page is a paid for option)</p>
<p>This is not necessarily a problem, you just need to keep track of the paid for
plugins and extensions so you can manage your monthly/yearly costs.</p>
<p>[1] https://en.wikipedia.org/wiki/Confluence_(software)</p>
<p>[2] https://en.wikipedia.org/wiki/Wiki</p>
<h1 id="why-use-confluence">Why use Confluence?</h1>
<p>Confluence is a large system with many functions.
In the past, I know confluence has been setup to be used in many areas of
of our IT team, including managing clinical trial build and progress etc.
We are not going to be using Confluence for this in the initial stages,
We are going to be concentrating on using confluence for two main functions:</p>
<ol>
<li>Knowledge Base (How-to Articles) [skip to section]</li>
<li>CPD training session [skip to section]</li>
</ol>
<p>Both these functions are interconnected.</p>
<h4 id="knowledge-base">Knowledge Base</h4>
<p>Provide material for reference on various internal IT tasks.</p>
<h4 id="cpd-training">CPD Training</h4>
<p>Our monthly training schedule for up an coming sessions.
Each session, should, ideally, have a corresponding wiki page.
This means that each CPD talk will have some reference material to look back on after the session.</p>
<h1 id="access-confluence">Access Confluence</h1>
<p>To access you navigate to your instances url, if hosted on Atlassian servers,
you url will mostly like be of the format <code class="language-plaintext highlighter-rouge">COMPANY-NAME.atlassian.net</code>.</p>
<p>This will land you on the main home page for your confluence system.</p>
<p>You can see there is a lot of pages, information and options.
Our particular confluence instance needs a bit of an overall, as we are not using it now,
as it was intended to be used when it was setup.</p>
<p>Some content need archiving, and things need a bit of a tidy up.
Rather than spend several months tidying this up, we will work with what we have for now,
and try to be more organised and systematic moving forwards.</p>
<p>In the mean time, I have created a list of important pages.</p>
<figure>
<img src="img/octru-it-pageslist.png" alt="" />
</figure>
<p>For the purposes of this introduction, and our evaluation of confluence,
we will be concentrating on 2 main pages (as mentioned above):</p>
<ol>
<li>Knowledge Base (How-to Articles)</li>
<li>CPD training session</li>
</ol>
<p>We also use the system for Compliance and Regulatory Standards</p>
<p>Compliance and Regulatory Standards;
table of reference of the standards we need to keep up to date with,
date of last reviewed and update alongside the related SOPs and Forms.</p>
<p>Covering the two main functions of confluence should get us up and running with confluence
to the point we can evaluate
if it is something we find useful, and if we want to continue using in the future.</p>
<h1 id="how-to-articles">How-to Articles</h1>
<p>In this part of the guide,
I am going to concentrate on the ‘How-to Articles’ section.</p>
<p>I will provide an overview, and also explain:</p>
<ul>
<li>How to add new articles to this section</li>
<li>How edit, update content and formatting.</li>
<li>How to use the tagging feature</li>
<li>Exporting as PDF</li>
</ul>
<p>Navigate to the url to access your How-to Articles page.</p>
<figure>
<img src="img/howto-pages.png" alt="" />
</figure>
<h2 id="add-new-article">Add new article</h2>
<p>On the main How-to Articles page, click the ‘Add how-to article’ button.</p>
<figure>
<img src="img/howto-pages.png" alt="" />
</figure>
<p>This will bring up a menu to enter the title of your new how to article,
along with some associated labels.</p>
<figure>
<img src="img/howto-addarticle.png" alt="" />
</figure>
<p>Although the format of the title and labels are up to the author,
if we follow a consistent format, it might be easier for us to use.</p>
<p>I have used the format, topic-101 to indicate an introduction article to the particular topic.
The ‘101’ guide will typically explain the basics, and provide enough information to get up and running.</p>
<p>The labels you can assign to your article, are again up to the author.
As long as you pick labels that are relevant to the article,
this should help up all navigate and search for content.</p>
<p>The title and labels can be edited after you create the article,
so you don’t need to get it perfect on your first go.</p>
<p>Note: by using this ‘How to article’ button to create the article,
you are essentially using a pre-built template. This template includes the ‘kb-how-to-article’ tag.</p>
<figure>
<img src="img/howto-kb-tag.png" alt="" />
</figure>
<h2 id="editing-and-formatting-content">Editing and formatting content</h2>
<p>Editing and formatting content using confluence is similar to using Microsoft word.</p>
<p>Once you have entered the edit mode
by clicking the pencil symbol from the menu.</p>
<figure>
<img src="img/howto-edit.png" alt="" />
</figure>
<p>You will see the top menu bar change to look more like Microsoft word.
You have options to bold, italic and colour text etc.</p>
<figure>
<img src="img/menu-doc-edit.png" alt="" />
</figure>
<p>There are a few components within the page that are automatically generated,
assuming you used the ‘add how to article’ button to create the page.</p>
<p>The components I will cover in this guide are:</p>
<ul>
<li>Table on Contents</li>
<li>Related Articles</li>
<li>More options</li>
<li>Markdown markup</li>
</ul>
<h3 id="table-on-contents">Table on contents</h3>
<p>This component automatically renders to display sections within the article.</p>
<figure>
<img src="img/howto-edit-toc.png" alt="" />
</figure>
<h3 id="related-articles">Related Articles</h3>
<p>The related articles section will use a list of user editable tags to show related articles.</p>
<!--
<figure>
<img src="img/howto-edit-relatedarticles/Large GIF (876x536).gif" alt="">
</figure>
-->
<h3 id="more-options">More options</h3>
<p>I invite you to play around and explore more options from the menu</p>
<figure>
<img src="img/howto-edit-additionaloptions.png" alt="" />
</figure>
<h3 id="markdown-markup">Markdown markup</h3>
<p>Note: advanced users who know how to use markdown markup, can use the majority of functionality
within the confluence edit page setup.</p>
<p>For example, the most commonly used for me is probably section, sub-section, sub-sub-section etc.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># Section
section one
## Sub-section
sub section one
### Sub-sub-section
sub sub section one
</code></pre></div></div>
<h2 id="tagging-feature">Tagging feature</h2>
<p>You can modify and add additional tags for an article at any time.
When creating new tags, have a quick check you are not creating duplicates.
The convention we have is lower kebab case labels.</p>
<figure>
<img src="img/howto-edit-labels/Large GIF (746x314).gif" alt="" />
</figure>
<h2 id="exporting-as-pdf">Exporting as PDF</h2>
<p>If you click an article, you and read and also export as PDF,
if you need to send the guide to someone who does not have access to confluence.</p>
<figure>
<img src="img/howto-exportpdf/Large GIF (722x164).gif" alt="" />
</figure>
<p>Once you have selected to export as PDF,
you will be taken to a page to download your PDF document.</p>
<figure>
<img src="img/howto-exportpdf-download.png" alt="" />
</figure>
<h1 id="cpd-sessions">CPD Sessions</h1>
<p>This is the page which contains information on our CPD sessions.</p>
<p>On this page, you can see a list of the types of CPD sessions that we will be
running as part of our department.</p>
<ul>
<li>Formal training</li>
<li>Demos</li>
<li>Event or conference (show and tell)</li>
<li>Problem solving sessions (take a common problem, hash out ideas for a solution)</li>
<li>New technology/technique discussion (found some new shiny tech?, we can discuss)</li>
</ul>
<figure>
<img src="img/types-of-cpd.png" alt="" />
</figure>
<p>This page also contains a table showing our CPD schedule.
This table contains the date, host and title of the CPD session taking place.
This table also contains the type of CPD, this is to try to help decide if this
is relevant for you or not.</p>
<figure>
<img src="img/sample-cpd-schedule.png" alt="" />
</figure>
<h2 id="addupdate-schedule">Add/update schedule</h2>
<p>If you would like to add or update the CPD schedule, you can do so by editing the page.</p>
<p>If you scroll to the top of the page, you should see a menu bar that looks like this</p>
<figure>
<img src="img/cpd-top-menu.png" alt="" />
</figure>
<p>There are various options; like, comment, watch, favourite.
We want to click the pencil symbol option to edit the page.</p>
<figure>
<img src="img/menu-edit.png" alt="" />
</figure>
<p>Once you have entered the edit mode, you will see the top menu bar change to look more like Microsoft word.
You have options to bold, italic and colour text etc.</p>
<figure>
<img src="img/menu-doc-edit.png" alt="" />
</figure>
<p>When updating or adding a new item to the CPD schedule,
there are 3 main advanced elements we need to be familiar with.</p>
<ol>
<li>Input date</li>
<li>Insert username</li>
<li>Link to document</li>
</ol>
<h2 id="input-date">Input date</h2>
<figure>
<img src="img/doc-insert-date/Large GIF (920x500).gif" alt="" />
</figure>
<h2 id="insert-username">Insert username</h2>
<figure>
<img src="img/doc-insert-username/Large GIF (916x286).gif" alt="" />
</figure>
<h2 id="link-to-document">Link to document</h2>
<figure>
<img src="img/doc-insert-doclink/Large GIF (924x372).gif" alt="" />
</figure>
<p>If you have any questions, please
<a id="link-contact" href="/contact/">get in touch</a>.
I would be glad to help out if I can.</p>Ayesh AlshukriWhat is the Impact of Reproducible Research?2021-03-30T09:00:00+01:002021-03-30T09:00:00+01:00https://ayeshalshukri.co.uk/category/discourse/reproducible-research<figure>
<img src="research.jpg" alt="Reproducible Research" />
</figure>
<p>This article presents the possible impact of reproducible research on your organisation.</p>
<h2 id="why-reproducable-research">Why reproducable research?</h2>
<p>My background is computer sciences
and I am passionate about using computers as a tool to make our lives
better no matter your level of expertise or background.</p>
<p>I have experience in developing scripts, automations, and
computational processes throughout my career that can make
research more:</p>
<ul>
<li>reusable</li>
<li>efficient</li>
<li>shareable</li>
</ul>
<p>I promote contributing to the wider community by sharing the digital assets that
I create and network with peer-groups to build and share expertise within a particular problem domain.</p>
<p>The data carpentries course would allow me to teach the skills I have gained over
my career to new audiences I would not have otherwise been able to effectively teach.</p>
<p>The goals of Reproducible Research are:</p>
<ul>
<li>coordinated approach to open research</li>
<li>promote reproducible research in all disciplines</li>
</ul>
<h2 id="fair-principals">FAIR Principals</h2>
<p>F - Findability</p>
<p>Metadata encoded in such a way to be machine-readable and descriptive so that data and digital assets are easy to keyword search and find.</p>
<p>A - Accessibility</p>
<p>I - Interoperability</p>
<p>R - Reuse of digital assets</p>
<p>https://www.go-fair.org/fair-principles/</p>
<h2 id="workshops">Workshops</h2>
<p>1) General</p>
<p>The first will be aimed at a general clinical trial based audience.
This would include; statisticians, management, directors, operational leads,
CI/PI, database officers, software developers, data analytics, regulatory and quality assurance.</p>
<p>I will present and discuss the FAIR principles and the advantages of promoting reproducible research
and the tools we have to help.
I also was to focus on the barriers and challenges of achieving more reproducible research
within a clinical and regulatory setting (ie, identify a training need, education opportunities,
resource or capacity constraints).</p>
<p>This workshop would have a particular focus on the efficiencies that can be gained,
and the overall contribution that producing more shareable and reusable digital assets can have
on our research output and contribution.</p>
<p>2) Practical</p>
<p>The second workshop would build upon the ideas and discussion of the first, but this time focus on a more practical interactive delivery.
This would ideally be suited to research practitioners.
This could range from statisticians and database officers who work on trial database builds, and well as research nurses and clinical and clerical staff who have a strong focus on data collection/analysis for clinical trials.</p>
<p>The delivery of the workshops will be focused on bringing the new knowledge and expertise from the data carpentries course to an audience that might not ordinarily have access to, or be aware of such tools and techniques.</p>
<p>This will be especially true with clinical, clerical, regulatory, and therapeutic staff within the clinical trials unit in which I am a part of.</p>
<p>A list of the therapeutic and research areas in which I am involved:</p>
<ul>
<li>surgery and interventional procedures (including anaesthetics,
surgical procedures, pre- and post-surgical care)</li>
<li>trauma, rehabilitation and conservative management</li>
<li>pharmaceuticals and imaging modalities</li>
</ul>
<h2 id="inclusion--diversity">Inclusion & Diversity</h2>
<p>I firmly believe that inclusion is a human right.
Practically, this means when providing an opportunity
we must remove any barrier to entry that would discriminate based on:</p>
<ul>
<li>race</li>
<li>gender</li>
<li>disability</li>
<li>medical needs etc.</li>
</ul>
<p>This inclusion and diversity will be of particular importance
for the workshops that I aim to deliver.
I will focus on providing equal access and removing intolerance.
In doing so, we will get a very diverse range of opinions and expertise from peoples
difference backgrounds, expertise, skill and seniority levels.
This will then in turn will help create a much richer learning experience for all involved.</p>
<h4 id="did-you-find-what-you-were-looking-for">Did you find what you were looking for?</h4>
<p>If not, please
<a id="link-contact" href="/contact/">get in touch</a>
I would be glad to help out if I can.</p>Ayesh Alshukri