Posted on Leave a comment

Redesigning Configuration Refresh for Azure App Configuration

Avatar

Overview

Since its inception, the .NET Core configuration provider for Azure App Configuration has provided the capability to monitor changes and sync them to the configuration within a running application. We recently redesigned this functionality to allow for on-demand refresh of the configuration. The new design paves the way for smarter applications that only refresh the configuration when necessary. As a result, inactive applications no longer have to monitor for configuration changes unnecessarily.
 

Initial design : Timer-based watch

In the initial design, configuration was kept in sync with Azure App Configuration using a watch mechanism which ran on a timer. At the time of initialization of the Azure App Configuration provider, users could specify the configuration settings to be updated and an optional polling interval. In case the polling interval was not specified, a default value of 30 seconds was used.

public static IWebHost BuildWebHost(string[] args)
{ WebHost.CreateDefaultBuilder(args) .ConfigureAppConfiguration((hostingContext, config) => { // Load settings from Azure App Configuration // Set up the provider to listen for changes triggered by a sentinel value var settings = config.Build(); string appConfigurationEndpoint = settings["AzureAppConfigurationEndpoint"]; config.AddAzureAppConfiguration(options => { options.ConnectWithManagedIdentity(appConfigurationEndpoint) .Use(keyFilter: "WebDemo:*") .WatchAndReloadAll(key: "WebDemo:Sentinel", label: LabelFilter.Null); }); settings = config.Build(); }) .UseStartup<Startup>() .Build();
}

For example, in the above code snippet, Azure App Configuration would be pinged every 30 seconds for changes. These calls would be made irrespective of whether the application was active or not. As a result, there would be unnecessary usage of network and CPU resources within inactive applications. Applications needed a way to trigger a refresh of the configuration on demand in order to be able to limit the refreshes to active applications. Then unnecessary checks for changes could be avoided.

This timer-based watch mechanism had the following fundamental design flaws.

  1. It could not be invoked on-demand.
  2. It continued to run in the background even in applications that could be considered inactive.
  3. It promoted constant polling of configuration rather than a more intelligent approach of updating configuration when applications are active or need to ensure freshness.
     

New design : Activity-based refresh

The new refresh mechanism allows users to keep their configuration updated using a middleware to determine activity. As long as the ASP.NET Core web application continues to receive requests, the configuration settings continue to get updated with the configuration store.

The application can be configured to trigger refresh for each request by adding the Azure App Configuration middleware from package Microsoft.Azure.AppConfiguration.AspNetCore in your application’s startup code.

public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{ app.UseAzureAppConfiguration(); app.UseMvc();
}

At the time of initialization of the configuration provider, the user can use the ConfigureRefresh method to register the configuration settings to be updated with an optional cache expiration time. In case the cache expiration time is not specified, a default value of 30 seconds is used.

public static IWebHost BuildWebHost(string[] args)
{ WebHost.CreateDefaultBuilder(args) .ConfigureAppConfiguration((hostingContext, config) => { // Load settings from Azure App Configuration // Set up the provider to listen for changes triggered by a sentinel value var settings = config.Build(); string appConfigurationEndpoint = settings["AzureAppConfigurationEndpoint"]; config.AddAzureAppConfiguration(options => { options.ConnectWithManagedIdentity(appConfigurationEndpoint) .Use(keyFilter: "WebDemo:*") .ConfigureRefresh((refreshOptions) => { // Indicates that all settings should be refreshed when the given key has changed refreshOptions.Register(key: "WebDemo:Sentinel", label: LabelFilter.Null, refreshAll: true); }); }); settings = config.Build(); }) .UseStartup<Startup>() .Build();
}

In order to keep the settings updated and avoid unnecessary calls to the configuration store, an internal cache is used for each setting. Until the cached value of a setting has expired, the refresh operation does not update the value. This happens even when the value has changed in the configuration store.  

Try it now!

For more information about Azure App Configuration, check out the following resources. You can find step-by-step tutorials that would help you get started with dynamic configuration using the new refresh mechanism within minutes. Please let us know what you think by filing issues on GitHub.

Overview: Azure App configuration
Tutorial: Use dynamic configuration in an ASP.NET Core app
Tutorial: Use dynamic configuration in a .NET Core app
Related Blog: Configuring a Server-side Blazor app with Azure App Configuration

Avatar

Software Engineer, Azure App Configuration

Follow    

Posted on Leave a comment

ASP.NET Core and Blazor updates in .NET Core 3.0 Preview 8

Avatar

Sourabh

.NET Core 3.0 Preview 8 is now available and it includes a bunch of new updates to ASP.NET Core and Blazor.

Here’s the list of what’s new in this preview:

  • Project template updates
    • Cleaned up top-level templates in Visual Studio
    • Angular template updated to Angular 8
    • Blazor templates renamed and simplified
    • Razor Class Library template replaces the Blazor Class Library template
  • Case-sensitive component binding
  • Improved reconnection logic for Blazor Server apps
  • NavLink component updated to handle additional attributes
  • Culture aware data binding
  • Automatic generation of backing fields for @ref
  • Razor Pages support for @attribute
  • New networking primitives for non-HTTP Servers
  • Unix domain socket support for the Kestrel Sockets transport
  • gRPC support for CallCredentials
  • ServiceReference tooling in Visual Studio
  • Diagnostics improvements for gRPC

Please see the release notes for additional details and known issues.

Get started

To get started with ASP.NET Core in .NET Core 3.0 Preview 8 install the .NET Core 3.0 Preview 8 SDK

If you’re on Windows using Visual Studio, install the latest preview of Visual Studio 2019.

.NET Core 3.0 Preview 8 requires Visual Studio 2019 16.3 Preview 2 or later

To install the latest Blazor WebAssembly template also run the following command:

dotnet new -i Microsoft.AspNetCore.Blazor.Templates::3.0.0-preview8.19405.7

Upgrade an existing project

To upgrade an existing an ASP.NET Core app to .NET Core 3.0 Preview 8, follow the migrations steps in the ASP.NET Core docs.

Please also see the full list of breaking changes in ASP.NET Core 3.0.

To upgrade an existing ASP.NET Core 3.0 Preview 7 project to Preview 8:

  • Update Microsoft.AspNetCore.* package references to 3.0.0-preview8.19405.7.
  • In Razor components rename OnInit to OnInitialized and OnInitAsync to OnInitializedAsync.
  • In Blazor apps, update Razor component parameters to be public, as non-public component parameters now result in an error.
  • In Blazor WebAssembly apps that make use of the HttpClient JSON helpers, add a package reference to Microsoft.AspNetCore.Blazor.HttpClient.
  • On Blazor form components remove use of Id and Class parameters and instead use the HTML id and class attributes.
  • Rename ElementRef to ElementReference.
  • Remove backing field declarations when using @ref or specify the @ref:suppressField parameter to suppress automatic backing field generation.
  • Update calls to ComponentBase.Invoke to call ComponentBase.InvokeAsync.
  • Update uses of ParameterCollection to use ParameterView.
  • Update uses of IComponent.Configure to use IComponent.Attach.
  • Remove use of namespace Microsoft.AspNetCore.Components.Layouts.

You should hopefully now be all set to use .NET Core 3.0 Preview 8.

Project template updates

Cleaned up top-level templates in Visual Studio

Top level ASP.NET Core project templates in the “Create a new project” dialog in Visual Studio no longer appear duplicated in the “Create a new ASP.NET Core web application” dialog. The following ASP.NET Core templates now only appear in the “Create a new project” dialog:

  • Razor Class Library
  • Blazor App
  • Worker Service
  • gRPC Service

Angular template updated to Angular 8

The Angular template for ASP.NET Core 3.0 has now been updated to use Angular 8.

Blazor templates renamed and simplified

We’ve updated the Blazor templates to use a consistent naming style and to simplify the number of templates:

  • The “Blazor (server-side)” template is now called “Blazor Server App”. Use blazorserver to create a Blazor Server app from the command-line.
  • The “Blazor” template is now called “Blazor WebAssembly App”. Use blazorwasm to create a Blazor WebAssembly app from the command-line.
  • To create an ASP.NET Core hosted Blazor WebAssembly app, select the “ASP.NET Core hosted” option in Visual Studio, or pass the --hosted on the command-line

Create a new Blazor app

dotnet new blazorwasm --hosted

Razor Class Library template replaces the Blazor Class Library template

The Razor Class Library template is now setup for Razor component development by default and the Blazor Class Library template has been removed. New Razor Class Library projects target .NET Standard so they can be used from both Blazor Server and Blazor WebAssembly apps. To create a new Razor Class Library template that targets .NET Core and supports Pages and Views instead, select the “Support pages and views” option in Visual Studio, or pass the --support-pages-and-views option on the command-line.

Razor Class Library for Pages and Views

dotnet new razorclasslib --support-pages-and-views

Case-sensitive component binding

Components in .razor files are now case-sensitive. This enables some useful new scenarios and improves diagnostics from the Razor compiler.

For example, the Counter has a button for incrementing the count that is styled as a primary button. What if we wanted a Button component that is styled as a primary button by default? Creating a component named Button in previous Blazor releases was problematic because it clashed with the button HTML element, but now that component matching is case-sensitive we can create our Button component and use it in Counter without issue.

Button.razor

<button class="btn btn-primary" @attributes="AdditionalAttributes" @onclick="OnClick">@ChildContent</button> @code { [Parameter] public EventCallback<UIMouseEventArgs> OnClick { get; set; } [Parameter] public RenderFragment ChildContent { get; set; } [Parameter(CaptureUnmatchedValues = true)] public IDictionary<string, object> AdditionalAttributes { get; set; }
}

Counter.razor

@page "/counter" <h1>Counter</h1> <p>Current count: @currentCount</p> <Button OnClick="IncrementCount">Click me</Button> @code { int currentCount = 0; void IncrementCount() { currentCount++; }
}

Notice that the Button component is pascal cased, which is the typical style for .NET types. If we instead try to name our component button we get a warning that components cannot start with a lowercase letter due to the potential conflicts with HTML elements.

Lowercase component warning

We can move the Button component into a Razor Class Library so that it can be reused in other projects. We can then reference the Razor Class Library from our web app. The Button component will now have the default namespace of the Razor Class Library. The Razor compiler will resolve components based on the in scope namespaces. If we try to use our Button component without adding a using statement for the requisite namespace, we now get a useful error message at build time.

Unrecognized component error

NavLink component updated to handle additional attributes

The built-in NavLink component now supports passing through additional attributes to the rendered anchor tag. Previously NavLink had specific support for the href and class attributes, but now you can specify any additional attribute you’d like. For example, you can specify the anchor target like this:

<NavLink href="my-page" target="_blank">My page</NavLink>

which would render:

<a href="my-page" target="_blank" rel="noopener noreferrer">My page</a>

Improved reconnection logic for Blazor Server apps

Blazor Server apps require a live connection to the server in order to function. If the connection or the server-side state associated with it is lost, then the the client will be unable to function. Blazor Server apps will attempt to reconnect to the server in the event of an intermittent connection loss and this logic has been made more robust in this release. If the reconnection attempts fail before the network connection can be reestablished, then the user can still attempt to retry the connection manually by clicking the provided “Retry” button.

However, if the server-side state associated with the connect was also lost (e.g. the server was restarted) then clients will still be unable to connect. A common situation where this occurs is during development in Visual Studio. Visual Studio will watch the project for file changes and then rebuild and restart the app as changes occur. When this happens the server-side state associated with any connected clients is lost, so any attempt to reconnect with that state will fail. The only option is to reload the app and establish a new connection.

New in this release, the app will now also suggest that the user reload the browser when the connection is lost and reconnection fails.

Reload prompt

Culture aware data binding

Data-binding support (@bind) for <input> elements is now culture-aware. Data bound values will be formatted for display and parsed using the current culture as specified by the System.Globalization.CultureInfo.CurrentCulture property. This means that @bind will work correctly when the user’s desired culture has been set as the current culture, which is typically done using the ASP.NET Core localization middleware (see Localization).

You can also manually specify the culture to use for data binding using the new @bind:culture parameter, where the value of the parameter is a CultureInfo instance. For example, to bind using the invariant culture:

<input @bind="amount" @bind:culture="CultureInfo.InvariantCulture" />

The <input type="number" /> and <input type="date" /> field types will by default use CultureInfo.InvariantCulture and the formatting rules appropriate for these field types in the browser. These field types cannot contain free-form text and have a look and feel that is controller by the browser.

Other field types with specific formatting requirements include datetime-local, month, and week. These field types are not supported by Blazor at the time of writing because they are not supported by all major browsers.

Data binding now also includes support for binding to DateTime?, DateTimeOffset, and DateTimeOffset?.

Automatic generation of backing fields for @ref

The Razor compiler will now automatically generate a backing field for both element and component references when using @ref. You no longer need to define these fields manually:

<button @ref="myButton" @onclick="OnClicked">Click me</button> <Counter @ref="myCounter" IncrementAmount="10" /> @code { void OnClicked() => Console.WriteLine($"I have a {myButton} and myCounter.IncrementAmount={myCounter.IncrementAmount}");
}

In some cases you may still want to manually create the backing field. For example, declaring the backing field manually is required when referencing generic components. To suppress backing field generation specify the @ref:suppressField parameter.

Razor Pages support for @attribute

Razor Pages now support the new @attribute directive for adding attributes to the generate page class.

For example, you can now specify that a page requires authorization like this:

@page
@attribute [Microsoft.AspNetCore.Authorization.Authorize] <h1>Authorized users only!<h1> <p>Hello @User.Identity.Name. You are authorized!</p>

New networking primitives for non-HTTP Servers

As part of the effort to decouple the components of Kestrel, we are introducing new networking primitives allowing you to add support for non-HTTP protocols.

You can bind to an endpoint (System.Net.EndPoint) by calling Bind on an IConnectionListenerFactory. This returns a IConnectionListener which can be used to accept new connections. Calling AcceptAsync returns a ConnectionContext with details on the connection. A ConnectionContext is similar to HttpContext except it represents a connection instead of an HTTP request and response.

The example below show a simple TCP Echo server hosted in a BackgroundService built using these new primitives.

public class TcpEchoServer : BackgroundService
{ private readonly ILogger<TcpEchoServer> _logger; private readonly IConnectionListenerFactory _factory; private IConnectionListener _listener; public TcpEchoServer(ILogger<TcpEchoServer> logger, IConnectionListenerFactory factory) { _logger = logger; _factory = factory; } protected override async Task ExecuteAsync(CancellationToken stoppingToken) { _listener = await _factory.BindAsync(new IPEndPoint(IPAddress.Loopback, 6000), stoppingToken); while (true) { var connection = await _listener.AcceptAsync(stoppingToken); // AcceptAsync will return null upon disposing the listener if (connection == null) { break; } // In an actual server, ensure all accepted connections are disposed prior to completing _ = Echo(connection, stoppingToken); } } public override async Task StopAsync(CancellationToken cancellationToken) { await _listener.DisposeAsync(); } private async Task Echo(ConnectionContext connection, CancellationToken stoppingToken) { try { var input = connection.Transport.Input; var output = connection.Transport.Output; await input.CopyToAsync(output, stoppingToken); } catch (OperationCanceledException) { _logger.LogInformation("Connection {ConnectionId} cancelled due to server shutdown", connection.ConnectionId); } catch (Exception e) { _logger.LogError(e, "Connection {ConnectionId} threw an exception", connection.ConnectionId); } finally { await connection.DisposeAsync(); _logger.LogInformation("Connection {ConnectionId} disconnected", connection.ConnectionId); } }
}

Unix domain socket support for the Kestrel Sockets transport

We’ve updated the default sockets transport in Kestrel to add support Unix domain sockets (on Linux, macOS, and Windows 10, version 1803 and newer). To bind to a Unix socket, you can call the ListenUnixSocket() method on KestrelServerOptions.

public static IHostBuilder CreateHostBuilder(string[] args) => Host.CreateDefaultBuilder(args) .ConfigureWebHostDefaults(webBuilder => { webBuilder .ConfigureKestrel(o => { o.ListenUnixSocket("/var/listen.sock"); }) .UseStartup<Startup>(); });

gRPC support for CallCredentials

In preview8, we’ve added support for CallCredentials allowing for interoperability with existing libraries like Grpc.Auth that rely on CallCredentials.

Diagnostics improvements for gRPC

Support for Activity

The gRPC client and server use Activities to annotate inbound/outbound requests with baggage containing information about the current RPC operation. This information can be accessed by telemetry frameworks for distributed tracing and by logging frameworks.

EventCounters

The newly introduced Grpc.AspNetCore.Server and Grpc.Net.Client providers now emit the following event counters:

  • total-calls
  • current-calls
  • calls-failed
  • calls-deadline-exceeded
  • messages-sent
  • messages-received
  • calls-unimplemented

You can use the dotnet counters global tool to view the metrics emitted.

dotnet counters monitor -p <PID> Grpc.AspNetCore.Server

ServiceReference tooling in Visual Studio

We’ve added support in Visual Studio that makes it easier to manage references to other Protocol Buffers documents and Open API documents.

ServiceReference

When pointed at OpenAPI documents, the ServiceReference experience in Visual Studio can generated typed C#/TypeScript clients using NSwag.

When pointed at Protocol Buffer (.proto) files, the ServiceReference experience will Visual Studio can generate gRPC service stubs, gRPC clients, or message types using the Grpc.Tools package.

SignalR User Survey

We’re interested in how you use SignalR and the Azure SignalR Service, and your opinions on SignalR features. To that end, we’ve created a survey we’d like to invite any SignalR customer to complete. If you’re interested in talking to one of the engineers from the SignalR team about your ideas or feedback, we’ve provided an opportunity to enter your contact information in the survey, but that information is not required. Help us plan the next wave of SignalR features by providing your feedback in the survey.

Give feedback

We hope you enjoy the new features in this preview release of ASP.NET Core and Blazor! Please let us know what you think by filing issues on GitHub.

Thanks for trying out ASP.NET Core and Blazor!

Avatar

Posted on Leave a comment

Azure SignalR Service now supports Event Grid!

Avatar

Ken

Since we GA’ed Azure SignalR Service in last September, serverless has become a very popular use case in Azure SignalR Service and is used by many customers. Unlike the traditional SignalR application which requires a server to host the hub, in serverless scenario no server is needed, instead you can directly send messages to clients through REST APIs or our management SDK which can easily be used in serverless code like Azure Functions.

Though there is a huge benefit which saves you the cost of maintaining the app server, the feature set in serverless scenario is limited. Since there is no real hub, it’s not possible to respond to client activities like client invocations or connection events. Without client events serverless use cases will be limited and we hear a lot of customers asking about this support. Today we’re excited to announce a new feature that enables Azure SignalR Service to publish client events to Azure Event Grid so that you can subscribe and respond to them.

How does it work?

Let’s first revisit how serverless scenario in Azure SignalR Service works.

  1. In serverless scenario, even you don’t have an app server, you still need to have a negotiate API so SignalR client can do the negotiation to get the url to SignalR service and a corresponding access token. Usually this can be done using an Azure Function.

  2. Client will then use the url and access token to connect to SignalR service.

  3. After clients are connected, you can send message to clients using REST APIs or service management SDK. If you are using Azure Functions, our SignalR Service binding does the work for you so you only need to return the messages as an output binding.

This flow is illustrated as step 1-3 in the diagram below:

Serverless workflow

What’s missing here is that there is no equivalent of OnConnected() and OnDisconnected() in serverless APIs so there is no way for the Azure function to know whether a client is connected or disconnected.

Now with Event Grid you’ll be able to get such events through an Event Grid subscription (as step 4 and 5 in the above diagram):

  1. When a client is connected/disconnected to SignalR service, service will publish this event to Event Grid.

  2. In Azure function you can have an Event Grid trigger and subscribe to such events, then Event Grid will send those events to the function (through a webhook).

How to use it?

It’s very simple to make your serverless application subscribe to SignalR connection events. Let’s use Azure function as an example.

  1. First you need to make sure your SignalR Service instance is in serverless mode. (Create a SignalR Service instance if you haven’t done so.)

    Enable serverless mode

  2. Create an Event Grid trigger in your function app.

    Create Event Grid trigger

  3. In the Event Grid trigger, add an Event Grid subscription.

    Add Event Grid Subscription

    Then select your SignalR Service instance.

    Select SignalR Service instance

Now you’re all set! Your function app is now able to get connection events from SignalR Service.

To test it, you just need to open a SignalR connection to the service. You can use the SignalR client in our sample repo, which contains a simple negotiate API implementation.

  1. Clone AzureSignalR-samples repo.

  2. Start the sample negotiation server.

    cd samples\Management\NegotiationServer
    set Azure__SignalR__ConnectionString=<connection_string>
    dotnet run
    
  3. Run SignalR client.

    cd samples\Management\SignalRClient
    dotnet run
    

    Open the function logs in Azure portal and you’ll see a connected event is sent to the function:

    Azure Function output

    If you stop the client you’ll also see a disconnected event is received.

Try it now!

This feature is now in public preview so feel free to try it out and let us know your feedback by filing issues on Github.

For more information about how to use Event Grid with SignalR Service, you can read this article or try this sample.

Avatar
Ken Chen

Principal Software Engineering Manager

Follow Ken   

Posted on Leave a comment

Extract images from URL in excel with PHP using PhpSpreadsheet

Last modified on August 7th, 2019 by Vincy.

There are various ways to extract images from a given URL. PHP contains built-in functions for extracting data including the images with a URL.

This article is for PHP code to extract images from URLs existing in an excel file.

I have used PhpSpreadsheet to read the URLs from an Excel file. Then, I created cURL script to extract images from the URL.

Extract Images from URL Read from Excel

PhpSpreadsheet library supports Excel read-write operations. It provides enormous features like formatting content, manipulating data and more. It has a rich set of built-in classes and thereby makes the development process easy.

Working with spreadsheets is a common need while handling excel data via programming. PhpSpreadsheet library reduces the developer’s effort on building applications with excel data handing.

We have already seen several examples of URL extract using PHP. Also, we have created code for getting video thumbnail from Youtube URL.

What is inside?

  1. Uses of extracting images from URL from Excel
  2. Advantages of PhpSpreadsheet Library
  3. Existing PHP libraries used to import-export
  4. File Structure
  5. About this example
  6. PHP code to load and extract image data
  7. Render extracted images in a gallery
  8. Database script
  9. Extract images from URL in excel using PhpSpreadsheet Output

Extracting of images from URL from an excel file will be helpful in many scenarios. Below list shows some scenarios.

  1. To import a large volume of images into your application’s media library.
  2. To migrate media files from one domain to another.
  3. To restore the Excel backup images into a database.
  4. To create a dynamic photo gallery without a database.

Advantages of PhpSpreadsheet Library

PhpSpreadsheet has many features and thereby has more advantages of using it.

  • It provides methods to prepare reports, charts, plans and more.
  • It has an option the read, write from a specified row, column and sheet of a spreadsheet document.
  • It is suitable for handling a large amount of data.
  • It helps to manage checklists, calendars, timesheets, schedules, proposal plans.
  • It provides security to protect spreadsheet data from editing.
  • It supports encryption to prevent the spreadsheet data from viewing. 

Existing PHP libraries used to import-export

There are many PHP libraries available in the market support spreadsheet data handling.

  • PortPHP supports import-export data between Excel, CSV and database storages. It has readers, writers and converters to process data exchange and manipulation.
  • The Spout is a PHP library used to read write spreadsheets in an efficient way. It supports three types of spreadsheets XLS, CSV, ODS.

File structure

Below screenshot shows the file structure of this example. The ExcelImportService class file is an integral part of this example. It loads PhpSpreadsheet library and covers all the operations related to the excel image extract.

The excel_template folder contains an input Excel file with image URLs. This example code loads this file to extract images from the URL.

Instead of using this fixed excel template, you can also allow users to choose an excel file. By adding a HTML form with a file input option user can choose their excel to explore extract.

Extract Images from URL from the Excel File Structure

About this example

This example loads an input Excel file in an Import service class. This sample excel file will contain image URLs.

In this example, I have used PhpSpreadsheet library to read the excel data. This library method helps to get the URLs and store into an array.

Then I iterate this URL array in a loop to extract the image data. I used PHP cURL script to extract images. In a previous tutorial, we have seen how to run PHP cURL script to extract content from a remote URL.

Finally, this code will store the extracted images into a directory and save the path to the database. In a previous article, we import excel data into a database without images. Also, we have seen examples to import data from CSV to a database.

PHP code to load and extract image data

This PHP code loads the ExcelImportService class to load and import image data from an excel.

This is the main PHP class created for this example. It handles all operations during the excel image extract.

<?php use \Phppot\ExcelImportService; require_once 'Class/ExcelImportService.php'; $excelImportService = new ExcelImportService(); $excelDataArray = $excelImportService->loadExcel(); if (! empty($excelDataArray)) { $isNewData = $excelImportService->importImages($excelDataArray); if ($isNewData) { $message = "Images extracted from excel successfully!"; } else { $message = "No new images found during the excel extract!"; } } $imageResult = $excelImportService->getAllImages(); ?> 

ExcelImportService.php

This class loads the PhpSpreadsheet library. It also has the DataSource instance in the class level.

The database access request from this class uses this instance. It is for saving the extracted image path to the database.

Note: Download PhpSpreadsheet library from Github without dependencies. Then run the get the dependencies via composer by using the following command.

composer require phpoffice/phpspreadsheet 

In this class, the loadExcel() function loads the input excel to read the URLs as an array. It returns this array to extract image blob via cURL request. 

The extractImage() function executes the cURL script. It gets the image resource data from the remote URL read from Excel. Then it writes the file into a target as specified in this example.

After putting the extracted images into a folder, then the code saves the to the image database table. The saveImagePath() method contains the insert query and parameters to invoke DataSource insert.

<?php namespace Phppot; use \Phppot\DataSource; require 'Vendor/PhpSpreadsheet/autoload.php'; class ExcelImportService { private $ds; function __construct() { require_once __DIR__ . './DataSource.php'; $this->ds = new DataSource(); } private function isUrlExist($url) { $query = 'SELECT * FROM tbl_images where remote_url = ?'; $paramType = 's'; $paramValue = array($url); $count = $this->ds->numRows($query, $paramType, $paramValue); return $count; } private function extractImage($url) { $path = pathinfo($url); $imageTargetPath = 'uploads/' . time() . $path['basename']; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_VERBOSE, 1); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_AUTOREFERER, false); curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1); curl_setopt($ch, CURLOPT_HEADER, 0); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0); // <-- important to specify curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0); // <-- important to specify $resultImage = curl_exec($ch); curl_close($ch); $fp = fopen($imageTargetPath, 'wb'); fwrite($fp, $resultImage); fclose($fp); $imageInfo["image_name"] = $path['basename']; $imageInfo["image_path"] = $imageTargetPath; return $imageInfo; } private function saveImagePath($imageInfo, $remoteUrl) { $query = "INSERT INTO tbl_images (image_name,image_path, remote_url) VALUES (?, ?, ?)"; $paramType = 'sss'; $paramValue = array($imageInfo["image_name"], $imageInfo["image_path"], $remoteUrl); $this->ds->insert($query, $paramType, $paramValue); } public function loadExcel() { //create directly an object instance of the IOFactory class, and load the xlsx file $xlsFile ='Excel_Template/imageURLs.xlsx'; $spreadsheet = \PhpOffice\PhpSpreadsheet\IOFactory::load($xlsFile); //read excel data and store it into an array $excelData = $spreadsheet->getActiveSheet()->toArray(null, true, true, true); $rowCount = count($excelData); $urlArray = array(); for($i=2;$i<$rowCount;$i++) { $url = $excelData[$i]['A']; if(!empty($url)) { $urlArray[] = $url; } } return $urlArray; } public function importImages($excelDataArray) { $isNewData = false; foreach($excelDataArray as $url) { $isUrlExist = $this->isUrlExist($url); if (empty($isUrlExist)) { $imageInfo = $this->extractImage($url); if(!empty($imageInfo)) { $this->saveImagePath($imageInfo, $url); } $isNewData = true; } } return $isNewData; } public function getAllImages() { $query = 'SELECT * FROM tbl_images'; $result = $this->ds->select($query); return $result; } }

DataSource.php

This is a common PHP class that we have used in many examples. It contains functions to execute the database operations planned for the example code. It establishes the database connection at its constructor.

Model classes used in our PHP examples load this class and instantiate it to access the database.

<?php namespace Phppot; /** * Generic datasource class for handling DB operations. * Uses MySqli and PreparedStatements. * * @version 2.3 */ class DataSource { // PHP 7.1.0 visibility modifiers are allowed for class constants. // when using above 7.1.0, declare the below constants as private const HOST = 'localhost'; const USERNAME = 'root'; const PASSWORD = ''; const DATABASENAME = 'phpsamples'; private $conn; /** * PHP implicitly takes care of cleanup for default connection types. * So no need to worry about closing the connection. * * Singletons not required in PHP as there is no * concept of shared memory. * Every object lives only for a request. * * Keeping things simple and that works! */ function __construct() { $this->conn = $this->getConnection(); } /** * If connection object is needed use this method and get access to it. * Otherwise, use the below methods for insert / update / etc. * * @return \mysqli */ public function getConnection() { $conn = new \mysqli(self::HOST, self::USERNAME, self::PASSWORD, self::DATABASENAME); if (mysqli_connect_errno()) { trigger_error("Problem with connecting to database."); } $conn->set_charset("utf8"); return $conn; } /** * To get database results * @param string $query * @param string $paramType * @param array $paramArray * @return array */ public function select($query, $paramType="", $paramArray=array()) { $stmt = $this->conn->prepare($query); if(!empty($paramType) && !empty($paramArray)) { $this->bindQueryParams($sql, $paramType, $paramArray); } $stmt->execute(); $result = $stmt->get_result(); if ($result->num_rows > 0) { while ($row = $result->fetch_assoc()) { $resultset[] = $row; } } if (! empty($resultset)) { return $resultset; } } /** * To insert * @param string $query * @param string $paramType * @param array $paramArray * @return int */ public function insert($query, $paramType, $paramArray) { $stmt = $this->conn->prepare($query); $this->bindQueryParams($stmt, $paramType, $paramArray); $stmt->execute(); $insertId = $stmt->insert_id; return $insertId; } /** * To execute query * @param string $query * @param string $paramType * @param array $paramArray */ public function execute($query, $paramType="", $paramArray=array()) { $stmt = $this->conn->prepare($query); if(!empty($paramType) && !empty($paramArray)) { $this->bindQueryParams($stmt, $paramType="", $paramArray=array()); } $stmt->execute(); } /** * 1. Prepares parameter binding * 2. Bind prameters to the sql statement * @param string $stmt * @param string $paramType * @param array $paramArray */ public function bindQueryParams($stmt, $paramType, $paramArray=array()) { $paramValueReference[] = & $paramType; for ($i = 0; $i < count($paramArray); $i ++) { $paramValueReference[] = & $paramArray[$i]; } call_user_func_array(array( $stmt, 'bind_param' ), $paramValueReference); } /** * To get database results * @param string $query * @param string $paramType * @param array $paramArray * @return array */ public function numRows($query, $paramType="", $paramArray=array()) { $stmt = $this->conn->prepare($query); if(!empty($paramType) && !empty($paramArray)) { $this->bindQueryParams($stmt, $paramType, $paramArray); } $stmt->execute(); $stmt->store_result(); $recordCount = $stmt->num_rows; return $recordCount; } } 

This is the HTML code to display the extracted images in the UI. I embed PHP code with this HTML to display the image path from the database dynamically.

The getAllImages() method fetches image results from the database. It returns an array of images extracted from the Excel. This array data iteration helps to render images in a gallery view.

<!doctype html> <html> <head> <link rel="stylesheet" type="text/css" href="CSS/style.css"> <title>Extract Images from URL in Excel using PHPSpreadSheet with PHP</title> </head> <body> <div id="gallery"> <div id="image-container"> <h2>Extract Images from URL in Excel using PHPSpreadSheet with PHP</h2> <?php if (! empty($message)) { ?> <div id="txtresponse"><?php echo $message; ?></div> <?php } ?> <ul id="image-list"> <?php if (! empty($imageResult)) { foreach ($imageResult as $k => $v) { ?> <li><img src="<?php echo $imageResult[$k]['image_path']; ?>" class="image-thumb" alt="<?php echo $imageResult[$k]['image_name'];?>"></li> <?php } } ?> </ul> </div> </div> </body> </html> 

After a successful image extract, this UI will acknowledge the user. It shows an appropriate message based on the image extract result.

If you extract an older excel that was already done, then the notification will say “No new images found”.

The following styles are used to present the extracted images in a gallery.

body { font-family: Arial; color: #212121; text-align: center; } #gallery { width: 1057px; margin: 0 auto; } #image-list { list-style-type: none; margin: 0; padding: 0; } #image-list li { margin: 10px 20px 10px 0px; display: inline-block; } #image-list li img { width: 250px; height: 155px; } #image-container { margin-bottom: 14px; } #txtresponse { padding: 10px 40px; border-radius: 3px; margin: 10px 0px 30px 0px; border: #ecdeaa 1px solid; color: #848483; background: #ffefb6; display: inline-block; } .btn-submit { padding: 10px 30px; background: #333; border: #E0E0E0 1px solid; color: #FFF; font-size: 0.9em; width: 100px; border-radius: 0px; cursor: pointer; position: absolute; } .image-thumb { background-color: grey; padding: 10px; } 

Database script

This SQL script is for creating the required database table in your environment. It has the create a statement of the tbl_images database table. This table is the storage the point to store the image local path.

Run this script before executing this example. You can also get the SQL script from the downloadable source code added with this article.

CREATE TABLE IF NOT EXISTS `tbl_images` ( `id` int(11) NOT NULL AUTO_INCREMENT, `image_name` varchar(50) NOT NULL, `image_path` varchar(50) NOT NULL, `date` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=263 ; 

The screenshot below shows the image gallery output. These images are from the uploads folder of this example. This is the local location to store the extracted images from the database.

This screen shows the user acknowledgment message above the gallery view. This acknowledgment varies based on the input excel file data.

Extract Images from URL from the Excel Output

If the input excel is too older and extracted already, then the below message will notify the user.

No New Images

Hope this article helps you to image extract from URLs present in excel. The example presented is the simplest way of demonstrating image extract from Excel.

Download

↑ Back to Top

Posted on Leave a comment

HttpRepl: A command-line tool for interacting with RESTful HTTP services

Angelos Petropoulos

Angelos

The ASP.NET team has built a command-line tool called HttpRepl. It lets you browse and invoke HTTP services in a similar way to working with files and folders. You give it a starting point (a base URL) and then you can execute commands like “dir” and “cd” to navigate your way around the API:

C:\> dotnet httprepl http://localhost:65369/
(Disconnected)~ set base http://localhost:65369
Using swagger metadata from http://localhost:65369/swagger/v1/swagger.json http://localhost:65369/~ dir
. []
Fruits [get|post]
People [get|post] http://localhost:65369/~ cd People
/People [get|post] http://localhost:65369/People~ dir
. [get|post]
.. []
{id} [get]

Once you have identified the API you are interested in, you can use all the typical HTTP verbs against it. Here is an example of calling GET on http://localhost:65369/People as a continuation from before:

http://localhost:65369/People~ get
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Wed, 24 Jul 2019 20:33:07 GMT
Server: Microsoft-IIS/10.0
Transfer-Encoding: chunked
X-Powered-By: ASP.NET [ { "id": 1, "name": "Scott Hunter" }, { "id": 0, "name": "Scott Hanselman" }, { "id": 2, "name": "Scott Guthrie" }
]

Right now HttpRepl is being shipped as a .NET Core Global Tool, which means all you have to do to get it is run the following command on a machine with the .NET Core SDK installed:

C:\> dotnet tool install -g Microsoft.dotnet-httprepl --version “3.0.0-*”

The ASP.NET team built HttpRepl for the purpose of exploring and testing APIs. The idea was to make the experience of exploring and testing APIs through a command-line more convenient. What do you think about HttpRepl and what other uses do you envision for it? We would love to hear your opinion, please leave us a comment below or visit the project on GitHub. And for those wondering, HttpRepl’s official ship date is expected to align with .NET Core 3.0 GA.

Configure Visual Studio Code to launch HttpRepl on debug

You can configure Visual Studio to launch HttpRepl when debugging (along with your web app) by creating a new launch configuration as follows:

"version": "0.2.0", "compounds": [ { "name": ".NET Core REPL", "configurations": [ ".NET Core Launch (web)", "httprepl" ] } ], "configurations": [ { "name": "httprepl", "type": "coreclr", "request": "launch", "program": "dotnet", "args": ["httprepl", "http://localhost:5000"], "cwd": "${workspaceFolder}", "stopAtEntry": false, "console": "integratedTerminal" }, { "name": ".NET Core Launch (web)", "type": "coreclr", "request": "launch", "preLaunchTask": "build", // If you have changed target frameworks, make sure to update the program path. "program": "${workspaceFolder}/bin/Debug/netcoreapp3.0/api.dll", "args": [], "cwd": "${workspaceFolder}", "stopAtEntry": false, // Enable launching a web browser when ASP.NET Core starts. For more information: https://aka.ms/VSCode-CS-LaunchJson-WebBrowser "serverReadyAction": { "action": "openExternally", "pattern": "^\\s*Now listening on:\\s+(https?://\\S+)" }, "env": { "ASPNETCORE_ENVIRONMENT": "Development" }, "sourceFileMap": { "/Views": "${workspaceFolder}/Views" } }

Configure Visual Studio for Windows to launch HttpRepl on F5

You can configure Visual Studio to automatically launch HttpRepl when you F5 a project with the following simple steps:

The .exe for HttpRepl on Windows can be found in the following location:

%USERPROFILE%\.dotnet\tools\dotnet-httprepl.exe

Don’t forget to select it from the menu after adding it:

Next time you F5 your project, Visual Studio will automatically launch HttpRepl with the appropriate base URL (same URL that would have been passed to a browser, controlled through launchsettings):

Note: We are currently working on integrating HttpRepl into Visual Studio, which will give you an out-of-the box and more refined experience.

Configure Visual Studio for Mac to launch HttpRepl as a Custom Tool

In Visual Studio for Mac, you can configure a Custom Tool to open a new Terminal window and start httprepl. To configure this, go to Tools>Edit Custom Tools…

This will bring you to the External Tools dialog where you can add a new tool. To get started click the Add button to add a new tool. Here you will configure a new tool to launch a new Terminal instance and start the httprepl tool. Fill in the dialog with the following values.

  • Title: dotnet httprepl
  • Command: osascript
  • Arguments: -e ‘tell application “Terminal” to activate’ -e ‘tell application “Terminal”
    to do script “dotnet-httprepl”‘
  • Working directory: ${ProjectDir}

See the image below showing this new tool:

After clicking OK a new tool will appear in the Tools menu. To test your application with httprepl, start your application with Run>Start Debugging (or Run>Start without Debugging) and then start the httprepl with the new tool in the Tools menu:

When you invoke the tool a new Terminal window should appear in the foreground. From here you can set the base url to that of the api that you would like to test with set base. For example, see the image below that shows set base was executed and a get request will be executed next:

Give us feedback

It’s not HTTPie, it’s not Curl, but it’s also not PostMan. It’s something that you run and stays running and its aware of its current context. We find this experience valuable, but ultimately what matters the most is what you think. Please let us know your opinion by leaving comments below or on GitHub.

Angelos Petropoulos

Posted on Leave a comment

SHADERed Free GLSL/HLSL Shader Editor

Released a few weeks back, SHADERed is a free and open source editing environment for developing shaders, both HLSL and GLSL.  SHADERed enables you to create shaders on the fly with a real-time view of the results.  Currently it is Windows only, but the code is currently being ported from D3D to OpenG+SDL so this could change in the future.

Features of SHADERed include:

  • instantly see changes
  • vertex, pixel and geometry shaders
  • render states
  • audio file support
  • load obj 3d model files
  • load your own textures into shaders
  • render results to render texture (or screen)
  • create and edit your own input variables
  • shader statistics
  • code editor with compilation and error reporting
  • custom themes and templates

SHADERed is available on Github here.  The code is available under the liberal MIT license.  Compiled binaries for Windows are available here.  Check the video below to see SHADERed in action.

[youtube https://www.youtube.com/watch?v=eBIjnGeIGPI&w=853&h=480]

Design Programming


Posted on Leave a comment

ASP.NET Core and Blazor updates in .NET Core 3.0 Preview 7

Daniel Roth

Daniel

.NET Core 3.0 Preview 7 is now available and it includes a bunch of new updates to ASP.NET Core and Blazor.

Here’s the list of what’s new in this preview:

  • Latest Visual Studio preview includes .NET Core 3.0 as the default runtime
  • Top level ASP.NET Core templates in Visual Studio
  • Simplified web templates
  • Attribute splatting for components
  • Data binding support for TypeConverters and generics
  • Clarified which directive attributes expect HTML vs C#
  • EventCounters
  • HTTPS in gRPC templates
  • gRPC Client Improvements
  • gRPC Metapackage
  • CLI tool for managing gRPC code generation

Please see the release notes for additional details and known issues.

Get started

To get started with ASP.NET Core in .NET Core 3.0 Preview 7 install the .NET Core 3.0 Preview 7 SDK

If you’re on Windows using Visual Studio, install the latest preview of Visual Studio 2019.

Note: .NET Core 3.0 Preview 7 requires Visual Studio 2019 16.3 Preview 1, which is being released later this week.

To install the latest client-side Blazor templates also run the following command:

dotnet new -i Microsoft.AspNetCore.Blazor.Templates::3.0.0-preview7.19365.7

Installing the Blazor Visual Studio extension is no longer required and it can be uninstalled if you’ve installed a previous version. Installing the Blazor WebAssembly templates from the command-line is now all you need to do to get them to show up in Visual Studio.

Upgrade an existing project

To upgrade an existing an ASP.NET Core app to .NET Core 3.0 Preview 7, follow the migrations steps in the ASP.NET Core docs.

Please also see the full list of breaking changes in ASP.NET Core 3.0.

To upgrade an existing ASP.NET Core 3.0 Preview 6 project to Preview 7:

  • Update Microsoft.AspNetCore.* package references to 3.0.0-preview7.19365.7.

That’s it! You should be ready to go.

Latest Visual Studio preview includes .NET Core 3.0 as the default runtime

The latest preview update for Visual Studio (16.3) includes .NET Core 3.0 as the default .NET Core runtime version. This means that if you install the latest preview of Visual Studio then you already have .NET Core 3.0. New project by default will target .NET Core 3.0

Top level ASP.NET Core templates in Visual Studio

The ASP.NET Core templates now show up as top level templates in Visual Studio in the “Create a new project” dialog.

ASP.NET Core templates

This means you can now search for the various ASP.NET Core templates and filter by project type (web, service, library, etc.) to find the one you want to use.

Simplified web templates

We’ve taken some steps to further simplify the web app templates to reduce the amount of code that is frequently just removed.

Specifically:

  • The cookie consent UI is no longer included in the web app templates by default.
  • Scripts and related static assets are now referenced as local files instead of using CDNs based on the current environment.

We will provide samples and documentation for adding these features to new apps as needed.

Attribute splatting for components

Components can now capture and render additional attributes in addition to the component’s declared parameters. Additional attributes can be captured in a dictionary and then “splatted” onto an element as part of the component’s rendering using the new @attributes Razor directive. This feature is especially valuable when defining a component that produces a markup element that supports a variety of customizations. For instance if you were defining a component that produces an <input> element, it would be tedious to define all of the attributes <input> supports like maxlength or placeholder as component parameters.

Accepting arbitrary parameters

To define a component that accepts arbitrary attributes define a component parameter using the [Parameter] attribute with the CaptureUnmatchedAttributes property set to true. The type of the parameter must be assignable from Dictionary<string, object>. This means that IEnumerable<KeyValuePair<string, object>> or IReadOnlyDictionary<string, object> are also options.

@code { [Parameter(CaptureUnmatchedAttributes = true)] Dictionary<string, object> Attributes { get; set; }
}

The CaptureUnmatchedAttributes property on [Parameter] allows that parameter to match all attributes that do not match any other parameter. A component can only define a single parameter with CaptureUnmatchedAttributes.

Using @attributes to render arbitrary attributes

A component can pass arbitrary attributes to another component or markup element using the @attributes directive attribute. The @attributes directive allows you to specify a collection of attributes to pass to a markup element or component. This is valuable because the set of key-value-pairs specified as attributes can come from a .NET collection and do not need to be specified in the source code of the component.

<input class="form-field" @attributes="Attributes" type="text" /> @code { [Parameter(CaptureUnmatchedAttributes = true)] Dictionary<string, object> Attributes { get; set; }
}

Using the @attributes directive the contents of the Attribute property get “splatted” onto the input element. If this results in duplicate attributes, then evaluation of attributes occurs from left to right. In the above example if Attributes also contained a value for class it would supersede class="form-field". If Attributes contained a value for type then that would be superseded by type="text".

Data binding support for TypeConverters and generics

Blazor now supports data binding to types that have a string TypeConverter. Many built-in framework types, like Guid and TimeSpan have a string TypeConverter, or you can define custom types with a string TypeConverter yourself. These types now work seamlessly with data binding:

<input @bind="guid" /> <p>@guid</p> @code { Guid guid;
}

Data binding also now works great with generics. In generic components you can now bind to types specified using generic type parameters.

@typeparam T <input @bind="value" /> <p>@value</p> @code { T value;
}

Clarified which directive attributes expect HTML vs C

In Preview 6 we introduced directive attributes as a common syntax for Razor compiler related features like specifying event handlers (@onclick) and data binding (@bind). In this update we’ve cleaned up which of the built-in directive attributes expect C# and HTML. Specifically, event handlers now expect C# values so a leading @ character is no longer required when specifying the event handler value:

@* Before *@
<button @onclick="@OnClick">Click me</button> @* After *@
<button @onclick="OnClick">Click me</button>

EventCounters

In place of Windows perf counters, .NET Core introduced a new way of emitting metrics via EventCounters. In preview7, we now emit EventCounters ASP.NET Core. You can use the dotnet counters global tool to view the metrics we emit.

Install the latest preview of dotnet counters by running the following command:

dotnet tool install --global dotnet-counters --version 3.0.0-preview7.19365.2

Hosting

The Hosting EventSourceProvider (Microsoft.AspNetCore.Hosting) now emits the following request counters:

  • requests-per-second
  • total-requests
  • current-requests
  • failed-requests

SignalR

In addition to hosting, SignalR (Microsoft.AspNetCore.Http.Connections) also emits the following connection counters:

  • connections-started
  • connections-stopped
  • connections-timed-out
  • connections-duration

To view all the counters emitted by ASP.NET Core, you can start dotnet counters and specify the desired provider. The example below shows the output when subscribing to events emitted by the Microsoft.AspNetCore.Hosting and System.Runtime providers.

dotnet counters monitor -p <PID> Microsoft.AspNetCore.Hosting System.Runtime

D8GX-5oV4AASKwM

New Package ID for SignalR’s JavaScript Client in NPM

The Azure SignalR Service made it easier for non-.NET developers to make use of SignalR’s real-time capabilities. A frequent question we would get from potential customers who wanted to enable their applications with SignalR via the Azure SignalR Service was “does it only work with ASP.NET?” The former identity of the ASP.NET Core SignalR – which included the @aspnet organization on NPM, only further confused new SignalR users.

To mitigate this confusion, beginning with 3.0.0-preview7, the SignalR JavaScript client will change from being @aspnet/signalr to @microsoft/signalr. To react to this change, you will need to change your references in package.json files, require statements, and ECMAScript import statements. If you’re interested in providing feedback on this move or to learn the thought process the team went through to make the change, read and/or contribute to this GitHub issue where the team engaged in an open discussion with the community.

New Customizable SignalR Hub Method Authorization

With Preview 7, SignalR now provides a custom resource to authorization handlers when a hub method requires authorization. The resource is an instance of HubInvocationContext. The HubInvocationContext includes the HubCallerContext, the name of the hub method being invoked, and the arguments to the hub method.

Consider the example of a chat room allowing multiple organization sign-in via Azure Active Directory. Anyone with a Microsoft account can sign in to chat, but only members of the owning organization should be able to ban users or view users’ chat histories. Furthermore, we might want to restrict certain functionality from certain users. Using the updated features in Preview 7, this is entirely possible. Note how the DomainRestrictedRequirement serves as a custom IAuthorizationRequirement. Now that the HubInvocationContext resource parameter is being passed in, the internal logic can inspect the context in which the Hub is being called and make decisions on allowing the user to execute individual Hub methods.

public class DomainRestrictedRequirement : AuthorizationHandler<DomainRestrictedRequirement, HubInvocationContext>, IAuthorizationRequirement
{ protected override Task HandleRequirementAsync(AuthorizationHandlerContext context, DomainRestrictedRequirement requirement, HubInvocationContext resource) { if (IsUserAllowedToDoThis(resource.HubMethodName, context.User.Identity.Name) && context.User != null && context.User.Identity != null && context.User.Identity.Name.EndsWith("@jabbr.net", StringComparison.OrdinalIgnoreCase)) { context.Succeed(requirement); } return Task.CompletedTask; } private bool IsUserAllowedToDoThis(string hubMethodName, string currentUsername) { return !(currentUsername.Equals("bob42@jabbr.net", StringComparison.OrdinalIgnoreCase) && hubMethodName.Equals("banUser", StringComparison.OrdinalIgnoreCase)); }
}

Now, individual Hub methods can be decorated with the name of the policy the code will need to check at run-time. As clients attempt to call individual Hub methods, the DomainRestrictedRequirement handler will run and control access to the methods. Based on the way the DomainRestrictedRequirement controls access, all logged-in users should be able to call the SendMessage method, only users who’ve logged in with a @jabbr.net email address will be able to view users’ histories, and – with the exception of bob42@jabbr.net – will be able to ban users from the chat room.

[Authorize]
public class ChatHub : Hub
{ public void SendMessage(string message) { } [Authorize("DomainRestricted")] public void BanUser(string username) { } [Authorize("DomainRestricted")] public void ViewUserHistory(string username) { }
}

Creating the DomainRestricted policy is as simple as wiring it up using the authorization middleware. In Startup.cs, add the new policy, providing the custom DomainRestrictedRequirement requirement as a parameter.

services .AddAuthorization(options => { options.AddPolicy("DomainRestricted", policy => { policy.Requirements.Add(new DomainRestrictedRequirement()); }); });

It must be noted that in this example, the DomainRestrictedRequirement class is not only a IAuthorizationRequirement but also it’s own AuthorizationHandler for that requirement. It is fine to split these into separate classes to separate concerns. Yet, in this way, there’s no need to inject the AuthorizationHandler during Startup, since the requirement and the handler are the same thing, there’s no need to inject the handler separately.

HTTPS in gRPC templates

The gRPC templates have been now been updated to use HTTPS by default. At development time, we continue the same certificate generated by the dotnet dev-certs tool and during production, you will still need to supply your own certificate.

gRPC Client Improvements

The managed gRPC client (Grpc.Net.Client) has been updated to target .NET Standard 2.1 and no longer depends on types present only in .NET Core 3.0. This potentially gives us the ability to run on other platforms in the future.

gRPC Metapackage

In 3.0.0-preview7, we’ve introduced a new package Grpc.AspNetCore that transitively references all other runtime and tooling dependencies required for building gRPC projects. Reasoning about a single package version for the metapackage should make it easier for developers to deal with as opposed multiple dependencies that version independently.

CLI tool for managing gRPC code generation

The new dotnet-grpc global tool makes it easier to manage protobuf files and their code generation settings. The global tool manages adding and removing protobuf files as well adding the required package references required to build and run gRPC applications.

Install the latest preview of dotnet-grpc by running the following command:

dotnet tool install --global dotnet-grpc --version 0.1.22-pre2

As an example, you can run following commands to generate a protobuf file and add it to your project for code generation. If you attempt this on a non-web project, we will default to generating a client and add the required package dependencies.

dotnet new proto -o .\Protos\mailbox.proto
dotnet grpc add-file .\Protos\mailbox.proto

Give feedback

We hope you enjoy the new features in this preview release of ASP.NET Core and Blazor! Please let us know what you think by filing issues on GitHub.

Thanks for trying out ASP.NET Core and Blazor!

Daniel Roth
Daniel Roth

Principal Program Manager, ASP.NET

Follow Daniel   

Posted on Leave a comment

Configuring a Server-side Blazor app with Azure App Configuration

Avatar

With .NET Core 3.0 Preview 6, we added authentication & authorization support to server-side Blazor apps. It only takes a matter of seconds to wire up an app to Azure Active Directory with support for single or multiple organizations. Once the project is created, it contains all the configuration elements in its appsettings.json to function. This is great, but in a team environment – or in a distributed topology – configuration files lead to all sorts of problems. In this post, we’ll take a look at how we can extract those configuration values out of JSON files and into an Azure App Configuration instance, where they can be used by other teammates or apps.

Setting up Multi-org Authentication

In the .NET Core 3.0 Preview 6 blog post we explored how to use the Individual User Accounts option in the authentication dialog to set up a Blazor app with ASP.NET Identity, so we won’t go into too much detail. Essentially, you click the Change link during project creation.

Click Change Auth during project creation

In this example I’ll be using an Azure Active Directory application to allow anyone with a Microsoft account to log into the app, so I’ll select Work or School Accounts and then select Cloud – Multiple Organizations in the Change Authentication dialog.

The Visual Studio add authentication dialog.

Once the project is created, my AzureAD configuration node contains the 3 key pieces of information my app’s code will need to authenticate against Azure Active Directory; my tenant URL, the client ID for the AAD app Visual Studio created for me during the project’s creation, and the callback URI so users can get back to my app once they’ve authenticated.

The appsettings.json inclusive of the settings.

Whilst this is conveniently placed here in my appsettings.json file, it’d be more convenient if I didn’t need any local configuration files. Having a centralized configuration-management solution would be easier to manage, as well as give me the ability to keep my config out of source control, should there come a point when things like connection strings need to be shared amongst developers.

Azure App Configuration

Azure App Configuration is a cloud-based solution for managing all of your configuration values. Once I have an Azure App Configuration instance set up in my subscription, adding the configuration settings is simple. By default, they’re hidden from view, but I can click Show Values or select an individual setting for editing or viewing.

The config values in Azure App Configuration

Convenient .NET Core IConfiguration Integration

The Azure App Configuration team has shipped a NuGet package containing extensions to ASP.NET and .NET Core that enable developers the ability of using the service, but without needing to change all your code that already makes use of IConfiguration. To start with, install the Microsoft.Extensions.Configuration.AzureAppConfiguration NuGet package.

Adding the NuGet Package for Azure App Configuration

You’ll need to copy the connection string from the Azure Portal to enable connectivity between your app and Azure App Configuration.

Copying the Azure App Configuration connection string

Once that value has been copied, you can use it with either dotnet user-secrets to configure your app, or using a debug-time environment variable. Though it seems like we’ve created yet one more configuration value to track, think about it this way: this is the only value you’ll have to set using an environment variable; all your other configuration can be set via Azure App Configuration in the portal.

Setting up the Azure App Configuration connection string in an environment variable

Using the Azure App Configuration Provider for .NET Core

Once the NuGet package is installed, the code to instruct my .NET Core code to use Azure App Configuration whenever it reads any configuration values from IConfiguration is simple. In Program.cs I’ll call the ConfigureAppConfiguration middleware method, then use the AddAzureAppConfiguration extension method to get the connection string from my ASPNETCORE_AzureAppConfigConnectionString environment variable. If the environment variable isn’t set, the call will noop and the other configuration providers will do the work.

This is great, because I won’t even need to change existing – or in this case, template-generated code – I just tell my app to use Azure App Configuration and I’m off to the races. The full update to Program.cs is shown below.

// using Microsoft.Extensions.Configuration.AzureAppConfiguration; public static IHostBuilder CreateHostBuilder(string[] args) => Host.CreateDefaultBuilder(args) .ConfigureAppConfiguration((hostingContext, config) => { config.AddAzureAppConfiguration(options => { var azureAppConfigConnectionString = hostingContext.Configuration["AzureAppConfigConnectionString"]; options.Connect(azureAppConfigConnectionString); }); }) .ConfigureWebHostDefaults(webBuilder => { webBuilder.UseStartup<Startup>(); });

When I run the app, it first reaches out to Azure App Configuration to get all the settings it needs to run and then works as if it were configured locally using appsettings.json. As long as my teammates or other services needing these values have the connection string to the Azure App Configuration instance holding the settings for the app, they’re good.

Running the authenticated app

Now, I can remove the configuration values entirely from the appsettings.json file. If I want to control the logging behavior using Azure App Configuration, I could move these left-over settings out, too. Even though I’ll be using Azure App Configuration as, the other providers are still there.

The appsettings.json with the settings removed.

Dynamic Re-loading

Log levels are a good example of how the Azure App Configuration service can enable dynamic reloading of configuration settings you might need to tweak frequently. By moving my logging configuration into Azure App Configuration, I can change the log level right in the portal. In Program.cs, I can use the Watch method to specify which configuration settings I’ll want to reload when they change.

public static IHostBuilder CreateHostBuilder(string[] args) => Host.CreateDefaultBuilder(args) .ConfigureAppConfiguration((hostingContext, config) => { config.AddAzureAppConfiguration(options => { var azureAppConfigConnectionString = hostingContext.Configuration["AzureAppConfigConnectionString"]; options.Connect(azureAppConfigConnectionString) .Watch("Logging:LogLevel:Default") .Watch("Logging:LogLevel:Microsoft") .Watch("Logging:LogLevel:Microsoft.Hosting.Lifetime"); }); }) .ConfigureWebHostDefaults(webBuilder => { webBuilder.UseStartup<Startup>(); });

The default load-time is 30 seconds, but now, should I need to turn up the volume on my logs to get a better view of what’s happening in my site, I don’t need to re-deploy or even stop my site. Simply changing the values in the portal will be enough – 30 seconds later the values will be re-loaded from Azure App Configuration and my logging will be more verbose.

Changing configuration values in the portal

Configuration Source Ordering

The JsonConfigurationSource configuration sources – those which load settings from appsettings.json and appsettings.{Environment}.json – are loaded during the call to CreateDefaultBuilder. So, by the time I call AddAzureAppConfiguration to load in the AzureAppConfigurationSource, the JSON file providers are already in the configuration sources list.

The importance of ordering is evident here; should I want to override the configuration values coming from Azure App Configuration with my local appsettings.json or appsettings.Development.json files, I’d need to re-order the providers in the call to ConfigureAppConfiguration. Otherwise, the JSON file values will be loaded first, then the last source (the one that will “win”) will be the Azure App Configuration source.

Try it Out

Any multi-node or microservice-based application topology benefits from centralized configuration, and teams benefit from it by not having to keep track of so many configuration settings, environment variables, and so on. Take a look over the Azure App Configuration documentation. You’ll see that there are a multitude of other features, like Feature Flags and dark deployment support. Then, create an instance and try wiring your existing ASP.NET Code up to read configuration values from the cloud.

Avatar
Brady Gaster

Senior Program Manager, ASP.NET Core

Follow    

Posted on Leave a comment

Forwarded Headers Middleware Updates in .NET Core 3.0 preview 6

Avatar

With the ASP.NET Core 2.1 release, we included UseHsts and UseHttpRedirection by default. These methods put a site into an infinite loop if deployed to an Azure Linux App Service, Azure Linux virtual machine (VM), or behind any other reverse proxy besides IIS. TLS is terminated by the reverse proxy, and Kestrel isn’t made aware of the correct request scheme.

OAuth and OIDC also fail in this configuration because they generate incorrect redirects. Calls to UseIISIntegration add and configure forwarded headers middleware when running behind IIS, but there’s no matching automatic configuration for Linux (Apache or Nginx integration). The fix for this issue is discussed in more detail in the doc article Forward the scheme for Linux and non-IIS reverse proxies.

Configuration-only Wire-up in Preview 6

With the updates in .NET Core 3 preview 6, you no longer need to call the middleware explicitly, as the host logic has been pre-wired to enable the Forwarded Headers Middleware by default as long as the ASPNETCORE_FORWARDEDHEADERS_ENABLED environment variable has been set to true. Turning on the Forwarded Headers Middleware is as simple as setting the ASPNETCORE_FORWARDEDHEADERS_ENABLED setting in the Azure Portal’s configuration blade for any App Service running on Linux or in a container.

Enabling the Forwarded Headers Middleware via config

Once this setting is set to true, the middleware starts working, and features dependent on Request.IsHttps resulting to true begin to function as expected.

Resolving the issue with ASP.NET Core 2.x Apps Today

If you’re currently building an ASP.NET Core 2.x app and want to run it on App Service for Linux now, there’s a workaround that will be future-proof when the updates come out for 3.0.

To forward the scheme from the proxy in non-IIS scenarios, add and configure Forwarded Headers Middleware. In Startup.cs, use the following code:

// using Microsoft.AspNetCore.HttpOverrides; public void ConfigureServices(IServiceCollection services)
{ if (string.Equals( Environment.GetEnvironmentVariable("ASPNETCORE_FORWARDEDHEADERS_ENABLED"), "true", StringComparison.OrdinalIgnoreCase)) { services.Configure<forwardedheadersoptions>(options => { options.ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto; // Only loopback proxies are allowed by default. // Clear that restriction because forwarders are enabled by explicit // configuration. options.KnownNetworks.Clear(); options.KnownProxies.Clear(); }); }
} public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{ app.UseForwardedHeaders();
}

If you enable your ASP.NET Core 2.x apps with this workaround today, when you’re ready to upgrade to 3.0, you’ll already have the right configuration setting in place.

Base Image Update

The base images used by the App Service team to streamline the creation of ASP.NET Core apps will soon be updated so that the ASPNETCORE_FORWARDEDHEADERS_ENABLED environment variable will be set to true. Once they’re updated, you won’t even need to explicitly set the environment variable; it’ll be enabled by default.

Try it Out

If you’re new to building ASP.NET Core apps using containers, the App Service options for Linux and Container-based hosting offer a great place to get started. The docs are loaded with guidance and examples, from how to Run a .NET Core app in App Service on Linux to accessing a SQL Server Database from an ASP.NET Core app running in App Service Linux.

Avatar
Brady Gaster

Senior Program Manager, ASP.NET Core

Follow    

Posted on Leave a comment

Humble Programming Book Bundle By Packt

Humble are currently running a new bundle absolutely loaded with programming books and videos by Packt Press.  The Humble Book Bundle: Programming by Packt.  The bundle includes books on C++, C#, Java, JavaScript and Go programming.

As always, Humble Bundles are split into different tiers.  The tiers of this bundle are:

1$

  • Understanding Software
  • C# 7 and .NET Core Cookbook
  • C++ Programming by Example
  • Go Cookbook
  • Learning JavaScript Data Structures and Algorithms

8$

  • Modern C++ Programming Cookbook
  • Advanced Go Programming in 7 Days (VIDEO)
  • Java 11 Cookbook
  • Modern JavaScript From the Beginning (VIDEO)
  • Python Programming Blueprints
  • Functional Python Programming
  • Python 3 Object-Oriented Programming

15$

  • Software Architect’s Handbook
  • Learning C# 8 and .NET Core 3.0 (VIDEO)
  • C++ High Performance
  • The Modern C++ Challenge
  • Mastering Go
  • Java 11 in 7 Days
  • Learning Java By Building Android Games
  • Hands-On Object Oriented Programming with Java 11 (VIDEO)
  • Learn Java 12 Programming
  • Python for Beginners (VIDEO)
  • Clean Code in Python
  • Expert Python Programming
  • C# 7.1 and .NET Core 2.0 – Modern Cross Platform Development

Purchasing using this link give you the opportunity to support this site, which if you do, thank you very much!

[youtube https://www.youtube.com/watch?v=HX8AIq5j5oI&w=853&h=480]

GameDev News Programming